Grape seems to work fairly well for adding jars to your classpath. It
also does a lot of other things such as fetching and dependency management.
e.g.
#!/home/robert/bin/groovy
import org.apache.commons.lang.StringUtils
#Grab(group='commons-lang', module='commons-lang', version='2.4')
def strings = ['Hello', 'Groovy', 'AVeryLongWord!', 'A simple sentence']
strings.each { String aString ->
println "$aString: ${StringUtils.abbreviate(aString,10)}"
}
Unfortunately if there is a jar on my filesystem that I want to
dynamically add to the filesystem then I have to resort to a much
uglier solution.
#!/home/robert/bin/groovy
def loader = this.class.classLoader.rootLoader
loader.addURL(new File("/home/robert/somejars/arithmetic-1.1.jar").toURI().toURL())
// can't use traditional package import
arithmeticMainClass = Class.forName("org.scharp.arithmetic.Main")
println "42 - 23 = " + arithmeticMainClass.subtract(42, 23)
// can't use "new" operator
myArithmeticObject = arithmeticMainClass.newInstance()
Is there a way to make grape grab a jar from the filesystem? If not,
can I somehow replicate what grape is doing in groovy/java?
I would like this solution to work for scripts that can be run by many users and many incompatible jars so adding jars to a common directory such as ~/.groovy/lib/ won't work.
I could create a local maven repository for local, jar libaries but
that seems like overkill.
This is how I solved this.
When Grape (Ivy) wants something it caches it under the ~/.groovy/grapes directory. All you need to do is just create your own ivy.xml file and throw your jar in there. I figured it out by just looking at some of the other artifacts donwloaded from maven. Here is a small example.....
We use Oracle here and I wanted it's jdbc jar file to be able to be 'Grabbed' by my Groovy scripts.
Unfortunately, I could not find any repository that had this jar on the web.
Step 1 : create directory ~/.groovy/grapes/com.oracle
Step 2 : create directory ~/.groovy/grapes/com.oracle/ojdbc6
Step 3 : create directory ~/.groovy/grapes/com.oracle/ojdbc6/jars
Step 4 : Get a copy of Oracle's ojdbc jar file and rename it. Our oracle version is 11.2.0.1.0 and we use Java6 so I got the locally installed ojdbc6.jar file and copied as ojdbc6-11.2.0.1.0.jar. This file I put into the directory created in the prior step.
Step 5 : create an ivy-11.2.0.1.0.xml config file.This file should be put into the directory created in step 2. For this step I heavily relied on examples from other artifacts grabbed. Any apache commons lib is a good example.
Here is my xml.
<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven"
>
<info organisation="com.oracle" module="ojdbc6" revision="11.2.0.1.0" status="release" publication="20130102153401">
<license name="" />
<description homepage="">Oracle ojdbc driver</description>
</info>
<configurations>
<conf name="default" visibility="public" description="" extends="runtime,master" />
<conf name="master" visibility="public" description="" />
<conf name="compile" visibility="public" description="" />
<conf name="provided" visibility="public" description="" />
<conf name="runtime" visibility="public" description="" extends="compile" />
<conf name="test" visibility="public" description="" extends="runtime" />
<conf name="system" visibility="public" description="" />
<conf name="sources" visibility="public" description="" />
<conf name="javadoc" visibility="public" description="" />
<conf name="optional" visibility="public" description="" />
</configurations>
<publications>
<artifact name="ojdbc6" type="jar" ext="jar" conf="master" />
</publications>
</ivy-module>
Now I can use this jar in my groovy scripts with the following....
#Grapes([
#GrabConfig(systemClassLoader=true),
#Grab('com.oracle:ojdbc6:11.2.0.1.0'),
])
import groovy.sql.*
To make things easy for deploying this grape to multiple servers I created a zip file that I could extract anywhere....
$ unzip -qql oracle_jdbc_groovy_grape.zip
0 06-11-2012 13:50 .groovy/grapes/com.oracle/
0 06-12-2012 14:17 .groovy/grapes/com.oracle/ojdbc6/
0 06-12-2012 14:17 .groovy/grapes/com.oracle/ojdbc6/jars/
2111220 06-11-2012 11:46 .groovy/grapes/com.oracle/ojdbc6/jars/ojdbc6-11.2.0.1.0.jar
2349 06-11-2012 11:50 .groovy/grapes/com.oracle/ojdbc6/ivy-11.2.0.1.0.xml
You can customize the ivy settings that Grape uses by creating a ~/.groovy/grapeConfig.xml file.
Here's an example how to use the local file system as repository:
<ibiblio name="local" root="file:${user.home}/.m2/repository/" m2compatible="true"/>
There's been some interest in adding this feature to grape but nothing serious. My guess is that it's unlikely that this will be added in the near future. (6-18 months)
According to this enhancement, Grapes will now also search your local Maven repo, along with Maven Central.
You can install any jar(s) in your local repo by:
running mvn install in your project
using mvn dependecy:get; see documentation and example
using mvn install:install-file; see documentation and example
Related
I am trying to get spark-submit (via GCP DataProc) to download maven dependencies from a GitHub packages repository.
Adding spark.jars.repositories=https://myuser:mytoken#maven.pkg.github.com/myorg/my-maven-packages-repo/ to the spark-submit command doesn't help...
The correct urls are accessed, but the files aren't downloaded (https://maven.pkg.github.com/myorg/my-maven-packages.repo/myorg/mylibrary/1.0.0/library-1.0.0.jar).
How can I get this working? (Without using uber-jars!)
Instead of adding:
spark.jars.repositories=https://myuser:mytoken#maven.pkg.github.com/myorg/my-maven-packages-repo/ to the spark-submit command
add:
--files=gs://my-bucket/github-ivysettings.xml
and
spark.jars.ivySettings=github-ivysettings.xml
upload the following file (github-ivysettings.xml) to the bucket:
<ivysettings>
<settings defaultResolver="default"/>
<include url="${ivy.default.settings.dir}/ivysettings-public.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-shared.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-local.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-main-chain.xml"/>
<include url="${ivy.default.settings.dir}/ivysettings-default-chain.xml"/>
<credentials
host="maven.pkg.github.com" realm="GitHub Package Registry"
username="myuser" passwd="mytoken"
/>
<resolvers>
<ibiblio
name="private-github"
m2compatible="true" useMavenMetadata="true" usepoms="true"
root="https://maven.pkg.github.com/myorg/my-maven-packages-repo/"
pattern="[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]"
/>
<chain name="default" returnFirst="true" checkmodified="true">
<resolver ref="local" />
<resolver ref="shared" />
<resolver ref="public" />
<resolver ref="private-github" />
</chain>
</resolvers>
</ivysettings>
This will preserve the current search order (local, shared, public) and then search in your private repository.
Note that the realm is important, so if you use this for a different private repository, change the host, the root and the realm.
1 of my 80 dll's has been updated. Previously we used Packagefortheweb 4.0 to build setup. Now we need to update our client tool. How can we use Packagefortheweb to do so.
Package for the Web is long since unsupported. If you have InstallShield you can make a compressed installation, or optionally a downloader, to do what you want. Note that PFTW primarily creates a compressed single-exe wrapper around a set of files including another exe, which (a) many tools can do these days, and (b) doesn't have any logic to update files on a target machine, so will need some sort of installation logic anyway.
use iexpress under Windows Winkey+R -> iexpress
It is real simple to use Package for the WEB under windows 10.
You need to create a manifest file with asInvoker privileges for the pftwwiz.exe and a requireAdministrator manifest for the stub32i.exe file.
Use the mt.exe tool to apply the manifest to this files.
Done!
The manifest code I use is this (the one for stub32i.exe), the one for the wizard just uses asInvoker:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<dependency>
<dependentAssembly>
<assemblyIdentity type="win32" name="Microsoft.Windows.Common-Controls" version="6.0.0.0" processorArchitecture="x86" publicKeyToken="6595b64144ccf1df" language="*"/>
</dependentAssembly>
</dependency>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedPrivileges>
<requestedExecutionLevel level="requireAdministrator" uiAccess="false"/>
</requestedPrivileges>
</security>
</trustInfo>
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings>
<dpiAware xmlns="http://schemas.microsoft.com/SMI/2005/WindowsSettings">true</dpiAware>
</windowsSettings>
</application>
<ms_compatibility:compatibility xmlns:ms_compatibility="urn:schemas-microsoft-com:compatibility.v1" xmlns="urn:schemas-microsoft-com:compatibility.v1">
<ms_compatibility:application xmlns:ms_compatibility="urn:schemas-microsoft-com:compatibility.v1">
<ms_compatibility:supportedOS xmlns:ms_compatibility="urn:schemas-microsoft-com:compatibility.v1" Id="{e2011457-1546-43c5-a5fe-008deee3d3f0}"/>
<ms_compatibility:supportedOS xmlns:ms_compatibility="urn:schemas-microsoft-com:compatibility.v1" Id="{35138b9a-5d96-4fbd-8e2d-a2440225f93a}"/>
<ms_compatibility:supportedOS xmlns:ms_compatibility="urn:schemas-microsoft-com:compatibility.v1" Id="{4a2f28e3-53b9-4441-ba9c-d69d4a4a6e38}"/>
<ms_compatibility:supportedOS xmlns:ms_compatibility="urn:schemas-microsoft-com:compatibility.v1" Id="{1f676c76-80e1-4239-95bb-83d0f6d0da78}"/>
<ms_compatibility:supportedOS xmlns:ms_compatibility="urn:schemas-microsoft-com:compatibility.v1" Id="{8e0f7a12-bfb3-4fe8-b9a5-48fd50a15a9a}"/>
</ms_compatibility:application>
</ms_compatibility:compatibility>
</assembly>
The batch file I use is this (I have copies with the addition original for this files):
#echo off
REM -- This batch file is needed to insert the external manifest file into the EXE, otherwise the program asks for administrative rights or fails to install
copy "pftwwiz (Original).exe" pftwwiz.exe
mt -manifest pftwwiz.manifest -outputresource:pftwwiz.exe
call signit pftwwiz.exe
copy pftwwiz.exe "pftwwiz (Patched).exe"
copy "stub32i (Original).exe" stub32i.exe
mt -manifest stub32i.manifest -outputresource:stub32i.exe
copy stub32i.exe "stub32i (Patched).exe"
I have a question based from this question
Replacing characters in Ant property
I want to build a variable (i can't use a property because i'm in a loop) that is pretty much StringA - StringB.
(maybe this is a misunderstanding of properties on my part but they can only be assigned once correct?)
I guess I could build a script function to calculate that, but my guess is that it must be possible to do it in an already existing function, probably something i'm missing.
this would be an example of the code
<for param="file">
<path>
<fileset dir="${mydir}" >
<include name="*.war"/>
</fileset>
</path>
<sequential>
<var name="undeploy_name" value="#{file} function_here ${mydir}" />
<JBossCLI port="${jboss.port.management-native}">
<undeploy namePattern="${undeploy_name}" />
</JBossCLI>
<deployToLiferay file="#{file}" />
</sequential>
</for>
in general I want to deploy several wars. this works fine when I run it once but if I want to make it re-runnable I need to undeploy them first.
I'm just a consumer of this interfaces, Ideally deployToLiferay would auto undeploy but it does not.
thanks for an feedback
edit: if I use something similar to what is define on the linked page i get:
<loadresource property="file-to-deploy">
<propertyresource name="#{file}"/>
<filterchain>
<tokenfilter>
<filetokenizer/>
<replacestring from="${mydir}" to=""/>
</tokenfilter>
</filterchain>
</loadresource>
10:52:49.541: * /data/contribution.xml:171: The following error occurred while executing this line:
10:52:49.541: * /data/contribution.xml:178: null doesn't exist
line 178 is my loadresource part
ANT is not a programming language. Personally I'd recommend embedding a scripting language like Groovy to process a group of files:
<target name="process-files" depends="resolve">
<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy" classpathref="build.path"/>
<fileset id="wars" dir="src/wars" includes="*.war"/>
<groovy>
project.references.wars.each {
ant.echo(message: "I want to do something with this ${it} file")
}
</groovy>
</target>
Example
├── build.xml
└── src
└── wars
├── app1.war
├── app2.war
└── app3.war
Example
process-files:
[echo] I want to do something with this /../src/wars/app1.war file
[echo] I want to do something with this /../src/wars/app2.war file
[echo] I want to do something with this /../src/wars/app3.war file
Update
The following working example shows how Apache ivy can be used to manage build dependencies. This is a capability that exists in other Java build tools like Maven.
<project name="demo" default="process-files" xmlns:ivy="antlib:org.apache.ivy.ant">
<available classname="org.apache.ivy.Main" property="ivy.installed"/>
<!--
==================
Normal ANT targets
==================
-->
<target name="process-files" depends="resolve">
<taskdef name="groovy" classname="org.codehaus.groovy.ant.Groovy" classpathref="build.path"/>
<fileset id="wars" dir="src/wars" includes="*.war"/>
<groovy>
project.references.wars.each {
ant.echo(message: "I want to do something with this ${it} file")
}
</groovy>
</target>
<!--
=============================
Dependency management targets
=============================
-->
<target name="resolve" depends="install-ivy">
<ivy:cachepath pathid="build.path">
<dependency org="org.codehaus.groovy" name="groovy-all" rev="2.4.7" conf="default"/>
</ivy:cachepath>
</target>
<target name="install-ivy" unless="ivy.installed">
<mkdir dir="${user.home}/.ant/lib"/>
<get dest="${user.home}/.ant/lib/ivy.jar" src="http://search.maven.org/remotecontent?filepath=org/apache/ivy/ivy/2.4.0/ivy-2.4.0.jar"/>
<fail message="Ivy has been installed. Run the build again"/>
</target>
</project>
I have GIT repo as below,
\main
\Module A
\Module B
\Shared
When i make a change on Module B, CCNET will make a build from Module A and then Module B,
I dont want CCNET to do this way.It will take a lot of time.
I just want ccnet build only changes on Module B.
Somebody please help me :
My 1 project in CCNET Config:
enter code here
<project name="Dashboard 5.0" queue="Dashboard_01" queuePriority="01" category="01">
<artifactDirectory>&pathToArtifactsDirectory;Dashboard 5.0\</artifactDirectory>
&workingDirectory;
<webURL>http://&buildServerAddress;/ccnet/server/&buildServerName;/project/Dashboard 5.0/ViewLatestBuildReport.aspx</webURL>
&modificationDelaySeconds;
<triggers>
<intervalTrigger seconds='30' buildCondition='IfModificationExists'/>
<scheduleTrigger time='03:00' buildCondition='ForceBuild' name='Scheduled'/>
<scheduleTrigger time='11:00' buildCondition='ForceBuild' name='Scheduled'/>
</triggers>
<state type="state" directory="&pathToStatesDirectory;Dashboard 5.0\" />
<sourcecontrol type="git">
<repository>&gitAddress;</repository>
<branch>master</branch>
<autoGetSource>true</autoGetSource>
<fetchSubmodules>false</fetchSubmodules>
<executable>C:\Program Files (x86)\Git\cmd\git.exe</executable>
<commitBuildModifications>false</commitBuildModifications>
<commitUntrackedFiles>false</commitUntrackedFiles>
<timeout>3000000</timeout>
</sourcecontrol>
<tasks>
<nant>
<executable>&pathToNantFile;</executable>
<baseDirectory>&pathToBuildScriptsDirectory;Dashboard 5.0\</baseDirectory>
<buildArgs>-D:projects_to_build=dashboard_framework</buildArgs>
<buildFile>cruise.build</buildFile>
<targetList>
<target>automate</target>
</targetList>
<buildTimeoutSeconds>3000</buildTimeoutSeconds>
</nant>
</tasks>
<publishers>
<merge>
<files>
<file>&pathDB5MainCheckoutDirectory;framework\build\test-reports\*Test.dll-results.xml</file>
<file>&pathDB5MainCheckoutDirectory;framework\build\test-reports\Test*.dll-results.xml</file>
<file>&pathDB5MainCheckoutDirectory;framework\build\test-reports\simian.xml</file>
</files>
</merge>
<xmllogger />
</publishers>
Any check in for Module A or Module B will cause the project node to start either way. What you are looking for is similar to what subversion update command on a sub folder does and Git is not intended to be this way. An alternative you can create a separate repository for each module. The build script or Nant task would have to be separated as well.
In Git, if you have several directories that are always checked out independently, then these are really two different projects and should live in two different repositories. You can merge them back together at a later point using Git Submodules
I had successfully extracted the compilation log present in my IDE into some one xml file very well. So in order to merge it I had mentioned in my ccnet.config file inside the publisher task using the <merge>
section.
But when I force my build, I am able to get the output.xml file correctly but an
error is thrown in ccnet.config window that it is unable to merge as this file is currently used by some other process.
Please see below:
[VSAT:ERROR] Publisher threw
exception:
ThoughtWorks.CruiseControl.Core.CruiseC
ontrolException: Unable to read the
contents of the file: C:
\ThreePartition\outp ut.xml --->
System.IO.IOException: The process
cannot access the file 'C:\ThreeP
artition\output.xml' because it is
being used by another process.
Can you suggest any method by which merging can be done successfully?
I have pasted the whole ccnet.config file below.
<project name="VSAT">
<sourcecontrol type="filtered">
<sourceControlProvider type="filesystem">
<repositoryRoot>C:\ThreePartition</repositoryRoot>
<autoGetSource>true</autoGetSource>
<ignoreMissingRoot>false</ignoreMissingRoot>
</sourceControlProvider>
<exclusionFilters>
<pathFilter>
<pattern>C:\ThreePartition\wrSbc750gx_ThreePartition\**</pattern>
</pathFilter>
<pathFilter>
<pattern>C:\ThreePartition\*.txt</pattern>
</pathFilter>
<pathFilter>
<pattern>C:\ThreePartition\*.xml</pattern>
</pathFilter>
</exclusionFilters>
</sourcecontrol>
<triggers>
<intervalTrigger name="continuous" seconds="240"
buildCondition="IfModificationExists" />
</triggers>
<tasks>
<nant>
<executable>C:\Nant-0.85\bin\NAnt.exe</executable>
<buildFile>nant.build</buildFile>
</nant>
</tasks>
<publishers>
<merge>
<files>
<file>C:\ThreePartition\output.xml</file>
</files>
</merge>
<xmllogger logDir="C:\Program Files\CruiseControl.NET\server\DF2.0-CI
\Logfiles" />
<email from="BuildAdmin#server.com"
mailhost="smtp.servermail.com" includeDetails="TRUE">
<users>
user name="Maddy" group="buildmaster"
address="Mymail#server.com"/>
</users>
<groups>
<group name="buildmaster" notification="always"/>
<group name="developers" notification="change"/>
</groups>
</email>
</publishers>
</project>
</cruisecontrol>
I had just placed the publishers section below for the better view
<publishers>
<merge>
<files>
<file>C:\ThreePartition\output.xml</file>
</files>
</merge>
<email from="BuildAdmin#server.com" mailhost="smtp.server.com" includeDetails="TRUE">
<users>
<user name="Maddy" group="buildmaster" address="Maddy.#server.com"/>
</users>
<groups> <group name="buildmaster" notification="always"/>
<group name="developers" notification="change"/>
</groups>
</email>
Some ideas:
Eliminate other obvious applications that would be writing to that file: other CCNet projects, other CCNet instances (e.g. are you maybe running the service and something from the command line?), or perhaps your source control.
If you're not attached to NAnt, try MSBuild and see if you get the same error. If all you're doing is compiling, you can pass the .sln or .csproj as a parameter directly to MSBuild.
Make sure you're on the latest version of CCNet - they regularly publish what I would consider fairly major bug fixes regularly.
What is says: the question is which process has open your xml while CC.net is trying to merge -- perhaps Process Explorer could be useful? Perhaps it works if you copy the xml output to a separate file and merge that.