I use this tutorial to connect to Gmail API: https://developers.google.com/gmail/api/quickstart/java
I would like to make a keyword in Katalon Studio, which depends on Gmail API.
I modified from sample code that line:
InputStream in = GmailQuickstart.class.getResourceAsStream(CREDENTIALS_FILE_PATH);
to this:
InputStream ins = new FileInputStream(CREDENTIALS_FILE_PATH);
JAR files are added, project is running and browser window is opened to get token. After successful authorization I got error message:
Caused by: java.lang.NoSuchMethodError:
com.google.api.client.http.HttpRequest.setResponseReturnRawInputStream(Z)Lcom/google/api/client/http/HttpRequest;
UPDATE: List of imported dependencies:
commons-codec-1.15.jar
commons-logging-1.2.jar
google-api-client-1.31.3.jar
google-api-client-extensions-1.6.0-beta.jar
google-api-client-jackson2-1.31.3.jar
google-api-client-java6-1.31.3.jar
google-api-services-gmail-v1-rev110-1.25.0.jar
google-http-client-1.39.1.jar
google-http-client-jackson2-1.39.1.jar
google-oauth-client-java6-1.31.4.jar
google-oauth-client-jetty-1.31.4.jar
guava-30.1.1-jre.jar
httpclient-4.5.13.jar
httpcore-4.4.14.jar
j2objc-annotations-1.3.jar
jackson-core-2.12.2.jar
jsr305-3.0.2.jar
https://docs.katalon.com/katalon-studio/docs/external-libraries.html#exclude-built-in-libraries
With the ability to remove built-in libraries stored in the .classpath file of a project folder, you can replace a built-in library with an external one for flexible libraries usage in a test project.
Requirements
An active Katalon Studio Enterprise license.
Katalon Studio version 7.8.
UPD:
i got katalon 7.9.1 and here how i was able to do it:
add the following class into KS project:
include/scripts/groovy/(default package)/GroovyBox.java
import groovy.lang.*;
import java.util.regex.Pattern;
import java.util.Map;
import java.util.List;
/** run groovy script in isolated classloader*/
public class GroovyBox {
GroovyShell gs;
public GroovyBox(ClassLoader parentCL, Pattern excludeClassPattern ) {
FilteredCL fcl = new FilteredCL(parentCL, excludeClassPattern);
gs = new GroovyShell(fcl);
}
public GroovyBox withClassPath(List<String> classPathList) {
GroovyClassLoader cl = gs.getClassLoader();
for(String cp: classPathList) cl.addClasspath(cp);
return this;
}
public Script parse(String scriptText) {
return gs.parse(scriptText);
}
public static class FilteredCL extends GroovyClassLoader{
Pattern filterOut;
public FilteredCL(ClassLoader parent,Pattern excludeClassPattern){
super(parent);
filterOut = excludeClassPattern;
}
#Override protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException{
if(filterOut.matcher(name).matches())throw new ClassNotFoundException("class not found "+ name);
return super.loadClass(name, resolve);
}
}
}
now add a test case - actually you can move code from test case into a class...
import ... /* all katalon imports here*/
assert method1() == 'HELLO WORLD'
def method1() {
def gb = new GroovyBox(this.getClass().getClassLoader().getParent(), ~/^com\.google\..*/)
def script = gb.parse('''
#Grab(group='com.google.api-client', module='google-api-client', version='1.31.3')
import com.google.api.client.http.HttpRequest
def c = HttpRequest.class
println( "methods execute:: "+c.methods.findAll{it.name=='execute'} )
println( "methods setResponseReturnRawInputStream:: "+c.methods.findAll{it.name=='setResponseReturnRawInputStream'} )
println greeting
return greeting.toUpperCase()
''')
script.setBinding([greeting:'hello world'] as Binding)
return script.run()
}
options to define external dependencies:
#Grab(...) as a first line of parsed script - loads all required dependencies from maven central (by default). for example #Grab(group='com.google.api-client', module='google-api-client', version='1.31.3') corresponds to this artifact.
sometimes you need to specify specific maven repository then add #GrabResolver(name='central', root='https://repo1.maven.org/maven2/')
if you want to specify local file dependencies then in the code above:
def gb = new GroovyBox(...).withClassPath([
'/path/to/lib1.jar',
'/path/to/lib2.jar'
])
Related
I'm going through this example but something about it is very confusing to me: https://www.testcookbook.com/book/groovy/jenkins/intro-testing-job-dsl.html
In this test, how/what is executing getJobFiles()? I don't see it being called anywhere. Is there some magic with jobFiles? Is specifying jobFiles somehow calling getJobFiles?
import javaposse.jobdsl.dsl.DslScriptLoader
import javaposse.jobdsl.plugin.JenkinsJobManagement
import org.junit.ClassRule
import org.jvnet.hudson.test.JenkinsRule
import spock.lang.Shared
import spock.lang.Specification
import spock.lang.Unroll
class JobScriptsSpec extends Specification {
#Shared
#ClassRule
JenkinsRule jenkinsRule = new JenkinsRule()
#Unroll
def 'test script #file.name'(File file) {
given:
def jobManagement = new JenkinsJobManagement(System.out, [:], new File('.'))
when:
new DslScriptLoader(jobManagement).runScript(file.text)
then:
noExceptionThrown()
where:
file << jobFiles
}
static List<File> getJobFiles() {
List<File> files = []
new File('jobs').eachFileRecurse {
if (it.name.endsWith('.groovy')) {
files << it
}
}
files
}
}
Edit
It seems like jobFiles does call getJobFiles() but I don't understand how. Is this a groovy or spock feature? I've been trying to research this but can finding anything explaining this in detail.
This is standard Groovy functionality. You can abbreviate any getter call like
def file = new File("sarek-test-parent/sarek-test-common/src/main/java")
println file.name // getName()
println file.parent // getParent()
println file.absolutePath // getAbsolutePath()
println file.directory // isDirectory()
java
sarek-test-parent\sarek-test-common\src\main
C:\Users\alexa\Documents\java-src\Sarek\sarek-test-parent\sarek-test-common\src\main\java
true
The same works for setters:
new Person().name = "John" // setName("John")
new Person().zipCode = "12345" // setZipCode("12345")
Actually the second link provided by jaco0646 explains it, just his mixing up this simple fact with data providers clouds the explanation.
Edit
When Groovy determines that jobFiles does not refer to any existing variable, it considers the name as a Groovy property, which then allows it to take advantage of the shortcut notation for accessing properties.
Problem:
I am trying to provide restriction (blacklisting ) all and allow only what I provided when we execute groovy using GroovyClassLoader
I am able to execute custom policy using with limited permission for GroovyClassLoader only.
Now I am trying to provide package restriction going to use as part of groovy execution. Let say If I allowed com.x.y this package if any package other then this used in groovy should throw SecurityException
I have tried to achieve the same with custom security manager and overriding the checkPackageAccess but didn't get success.
public TestSecurityManager extends SecurityManager{
List<String> whiteListedPkgList;
public void checkPackageAccess(String pkg){
if(!pkg.startWith(any of given white list pkg)){
throw new SecurityException("Access Denied");
}
//If package not belong to whilelisted package list throw security exception
}
}
When I tried using above approach we need to provide all packages for execution like com, java etc instead of java.nio.file. in whitelist list
UPDATE
If we try to allow package like com.x.y using white list comparison using start with check access pkg, it will allow that package but later on it will throw security exception for com package.
Could any one help for the same how we can achieve it ?
Thanks in advance
If you're able to, instead of using Java's SecurityManager, using the Groovy DSL features you can more easily achieve this.
See https://www.groovy-lang.org/dsls.html#_secure_ast_customizer
Example:
import groovy.transform.CompileStatic
import org.codehaus.groovy.control.CompilerConfiguration
import org.codehaus.groovy.control.customizers.CompilationCustomizer
import org.codehaus.groovy.control.customizers.SecureASTCustomizer
#CompileStatic
class Main {
static final CompilationCustomizer scz = new SecureASTCustomizer().with {
closuresAllowed = false // user will not be able to write closures
methodDefinitionAllowed = false // user will not be able to define methods
importsWhitelist = [] // empty whitelist means imports are disallowed
staticImportsWhitelist = [] // same for static imports
staticStarImportsWhitelist = ['java.lang.Math'] // only java.lang.Math is allowed
constantTypesClassesWhiteList = [
Integer,
Float,
Long,
Double,
BigDecimal,
Integer.TYPE,
Long.TYPE,
Float.TYPE,
Double.TYPE,
Object,
String,
].asImmutable() as List<Class>
// method calls are only allowed if the receiver is of one of those types
// be careful, it's not a runtime type!
receiversClassesWhiteList = [
Math,
Integer,
Float,
Double,
Long,
BigDecimal,
PrintStream,
Object,
].asImmutable() as List<Class>
it
}
static void main(args) {
def configuration = new CompilerConfiguration()
configuration.addCompilationCustomizers(scz)
// evaluate sandboxed code
new GroovyShell(configuration).evaluate(
""" println 'hello world' """)
}
}
If all you need is to whitelist certain classes, you can also try writing your own class loader and using that to evalute the sandboxed script:
class MyClassLoader extends ClassLoader {
Set<String> whiteListPackages = [
'java.lang.', 'java.util.', 'groovy.', 'org.codehaus.groovy.', 'Script'
]
MyClassLoader(ClassLoader parent) {
super(parent)
}
#Override
protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
if (!whiteListPackages.any { okPkg -> name.startsWith(okPkg) }) {
throw new ClassNotFoundException('Access is forbidden')
}
return super.loadClass(name, resolve)
}
}
def shell = new GroovyShell(new MyClassLoader(GroovySystem.classLoader))
// evaluate the script with our own classloader
shell.evaluate('''
println 'hello'
println([1,2,3])
// This line throws an error because the `java.net` package is not whitelisted
println(new URL('https://groovy-lang.org'))
''')
I'm running into a problem with GroovyScriptEngine - it seems not to be able to work with inner classes. Anyone know whether there's some limitation in GroovyScriptEngine or a workaround?
I have a directory with these two files:
// MyClass.groovy
public class MyClass {
MyOuter m1;
MyOuter.MyInner m2;
}
and
// MyOuter.groovy
public class MyOuter {
public static class MyInner {}
}
I have a following test class:
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import groovy.util.GroovyScriptEngine;
public class TestGroovyScriptEngine {
public static void main(String[] args) throws MalformedURLException, ClassNotFoundException {
final File myGroovySourceDir = new File("C:/MyGroovySourceDir");
final URL[] urls = { myGroovySourceDir.toURL() };
GroovyScriptEngine groovyScriptEngine = new GroovyScriptEngine(urls,
Thread.currentThread().getContextClassLoader());
Class<?> clazz = groovyScriptEngine.getGroovyClassLoader().loadClass("MyClass");
}
}
When I run it I get the following compilation error:
Exception in thread "main" org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
C:\MyGroovySourceDir\MyClass.groovy: 3: unable to resolve class MyOuter.MyInner
# line 3, column 2.
MyOuter.MyInner m2;
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:311)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:983)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:633)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:582)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:354)
at groovy.lang.GroovyClassLoader.access$300(GroovyClassLoader.java:87)
at groovy.lang.GroovyClassLoader$5.provide(GroovyClassLoader.java:323)
at groovy.lang.GroovyClassLoader$5.provide(GroovyClassLoader.java:320)
at org.codehaus.groovy.runtime.memoize.ConcurrentCommonCache.getAndPut(ConcurrentCommonCache.java:147)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:318)
at groovy.util.GroovyScriptEngine$ScriptClassLoader.doParseClass(GroovyScriptEngine.java:248)
at groovy.util.GroovyScriptEngine$ScriptClassLoader.parseClass(GroovyScriptEngine.java:235)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:307)
at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:811)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:767)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:836)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:824)
I would have expected a "clean compile", but the inner class seems to be causing problems.
My groovy classes compile fine at the command line using groovyc, or in Eclipse.
You have faced an edge case here. To clarify what happens let's define the initial conditions:
you have a Java (or Groovy) class that gets executed inside JVM
you have two Groovy classes that get loaded outside of the JVM
The problem you have described does not exist if you put these two Groovy classes inside the same path you execute your Java class from - in this case IDE takes care to compile these Groovy classes and put them to the classpath of a JVM that gets started to run your Java test class.
But this is not your case and you are trying to load these two Groovy classes outside the running JVM using GroovyClassLoader (which extends URLClassLoader btw). I will try to explain in the simplest possible words what happened that adding field of type MyOuter does not throw any compilation error, but MyOuter.MyInner does.
When you execute:
Class<?> clazz = groovyScriptEngine.getGroovyClassLoader().loadClass("MyClass");
Groovy class loader goes to script file lookup part, because it was not able to find MyClass in the current classpath. This is the part responsible for it:
// at this point the loading from a parent loader failed
// and we want to recompile if needed.
if (lookupScriptFiles) {
// try groovy file
try {
// check if recompilation already happened.
final Class classCacheEntry = getClassCacheEntry(name);
if (classCacheEntry != cls) return classCacheEntry;
URL source = resourceLoader.loadGroovySource(name);
// if recompilation fails, we want cls==null
Class oldClass = cls;
cls = null;
cls = recompile(source, name, oldClass);
} catch (IOException ioe) {
last = new ClassNotFoundException("IOException while opening groovy source: " + name, ioe);
} finally {
if (cls == null) {
removeClassCacheEntry(name);
} else {
setClassCacheEntry(cls);
}
}
}
Source: src/main/groovy/lang/GroovyClassLoader.java#L733-L753
Here URL source = resourceLoader.loadGroovySource(name); it loads the full file URL to the source file and here cls = recompile(source, name, oldClass); it executes class compilation.
There are several phases involved in Groovy class compilation. One of them is Phase.SEMANTIC_ANALYSIS which analyses class fields and their types for instance. At this point ClassCodeVisitorSupport executes visitClass(ClassNode node) for MyClass class and following line
node.visitContents(this);
starts class contents processing. If we take a look at the source code of this method:
public void visitContents(GroovyClassVisitor visitor) {
// now let's visit the contents of the class
for (PropertyNode pn : getProperties()) {
visitor.visitProperty(pn);
}
for (FieldNode fn : getFields()) {
visitor.visitField(fn);
}
for (ConstructorNode cn : getDeclaredConstructors()) {
visitor.visitConstructor(cn);
}
for (MethodNode mn : getMethods()) {
visitor.visitMethod(mn);
}
}
Source: src/main/org/codehaus/groovy/ast/ClassNode.java#L1066-L108
we will see that it analyses and process class properties, fields, constructors and methods. At this phase it resolves all types defined for these elements. It sees that there are two properties m1 and m2 with types MyOuter and MyOuter.MyInner accordingly, and it executes visitor.visitProperty(pn); for them. This method executes the one we are looking for - resolve()
private boolean resolve(ClassNode type, boolean testModuleImports, boolean testDefaultImports, boolean testStaticInnerClasses) {
resolveGenericsTypes(type.getGenericsTypes());
if (type.isResolved() || type.isPrimaryClassNode()) return true;
if (type.isArray()) {
ClassNode element = type.getComponentType();
boolean resolved = resolve(element, testModuleImports, testDefaultImports, testStaticInnerClasses);
if (resolved) {
ClassNode cn = element.makeArray();
type.setRedirect(cn);
}
return resolved;
}
// test if vanilla name is current class name
if (currentClass == type) return true;
String typeName = type.getName();
if (genericParameterNames.get(typeName) != null) {
GenericsType gt = genericParameterNames.get(typeName);
type.setRedirect(gt.getType());
type.setGenericsTypes(new GenericsType[]{ gt });
type.setGenericsPlaceHolder(true);
return true;
}
if (currentClass.getNameWithoutPackage().equals(typeName)) {
type.setRedirect(currentClass);
return true;
}
return resolveNestedClass(type) ||
resolveFromModule(type, testModuleImports) ||
resolveFromCompileUnit(type) ||
resolveFromDefaultImports(type, testDefaultImports) ||
resolveFromStaticInnerClasses(type, testStaticInnerClasses) ||
resolveToOuter(type);
}
Source: src/main/org/codehaus/groovy/control/ResolveVisitor.java#L343-L378
This method gets executed for both MyOuter and MyOuter.MyInner classes. It is worth mentioning that class resolving mechanism only checks if given class is available in the classpath and it does not load or parse any classes. That is why MyOuter gets recognized when this method reaches resolveToOuter(type). If we take a quick look at its source code we will understand why it works for this class:
private boolean resolveToOuter(ClassNode type) {
String name = type.getName();
// We do not need to check instances of LowerCaseClass
// to be a Class, because unless there was an import for
// for this we do not lookup these cases. This was a decision
// made on the mailing list. To ensure we will not visit this
// method again we set a NO_CLASS for this name
if (type instanceof LowerCaseClass) {
classNodeResolver.cacheClass(name, ClassNodeResolver.NO_CLASS);
return false;
}
if (currentClass.getModule().hasPackageName() && name.indexOf('.') == -1) return false;
LookupResult lr = null;
lr = classNodeResolver.resolveName(name, compilationUnit);
if (lr!=null) {
if (lr.isSourceUnit()) {
SourceUnit su = lr.getSourceUnit();
currentClass.getCompileUnit().addClassNodeToCompile(type, su);
} else {
type.setRedirect(lr.getClassNode());
}
return true;
}
return false;
}
Source: src/main/org/codehaus/groovy/control/ResolveVisitor.java#L725-L751
When Groovy class loader tries to resolve MyOuter type name it reaches
lr = classNodeResolver.resolveName(name, compilationUnit);
which locates script with a name MyOuter.groovy and it creates a SourceUnit object associated with this script file name. It is simply something like saying "OK, this class is not in my classpath at the moment, but there is a source file I can see that once compiled it will provide a valid type of name MyOuter". This is why it finally reaches:
currentClass.getCompileUnit().addClassNodeToCompile(type, su);
where currentClass is an object associated with MyClass type - it adds this source unit to MyClass compilation unit, so it gets compiled with the MyClass class. And this is the point where resolving
MyOuter m1
class property ends.
In the next step it picks MyOuter.MyInner m2 property and it tries to resolve its type. Keep in mind - MyOuter got resolved correctly, but it didn't get loaded to the classpath, so it's static inner class does not exist in any scope, yet. It goes through the same resolving strategies as MyOuter, but any of them works for MyOuter.MyInner class. And this is why ResolveVisitor.resolveOrFail() eventually throws this compilation exception.
Workaround
OK, so we know what happens, but is there anything we can do about it? Luckily, there is a workaround for this problem. You can run your program and load MyClass successfully only if you load MyOuter class to Groovy script engine first:
import java.io.File;
import java.net.MalformedURLException;
import java.net.URL;
import groovy.util.GroovyScriptEngine;
public class TestGroovyScriptEngine {
public static void main(String[] args) throws MalformedURLException, ClassNotFoundException {
final File myGroovySourceDir = new File("C:/MyGroovySourceDir");
final URL[] urls = { myGroovySourceDir.toURL() };
GroovyScriptEngine groovyScriptEngine = new GroovyScriptEngine(urls,
Thread.currentThread().getContextClassLoader());
groovyScriptEngine.getGroovyClassLoader().loadClass("MyOuter");
Class<?> clazz = groovyScriptEngine.getGroovyClassLoader().loadClass("MyClass");
}
}
Why does it work? Well, semantic analysis of MyOuter class does not cause any problems, because all types are known at this stage. This is why loading MyOuter class succeeds and it results in Groovy script engine instance knows what MyOuter and MyOuter.MyInner types are. So when you next load MyClass from the same Groovy script engine it will apply different resolving strategy - it will find both classes available to the current compilation unit and it wont have to resolve MyOuter class based on its Groovy script file.
Debugging
If you want to examine this use case better it is worth to run a debugger and see analyze what happens at the runtime. You can create a breakpoint at line 357 of ResolveVisitor.java file for instance, to see described scenario in action. Keep in mind one thing though - resolveFromDefaultImports(type, testDefaultImports) will try to lookup MyClass and MyOuter classes by applying default packages like java.util, java.io, groovy.lang etc. This resolve strategy kicks in before resolveToOuter(type) so you have to patiently jump through them. But it is worth it to see and get a better understanding about how things work. Hope it helps!
I'm writing a little test framework for a site using Geb. As part of my reporting functionality, I'd like to be able to specify at runtime where the reportsDir lives. I'm no developer so please excuse any omissions in this question.
Everything I've read so far suggests that this can only be set via the project's configuration or the build adaptor. However, Geb's Configuration class has a setReportsDir method which I can access from my browser object:
def currentConfig = pageBrowser.getConfig()
def reportLocation = "target/runtime_reports_dir"
def reportFile = new File(reportLocation)
reportFile.mkdirs()
File newTarget = new File(reportLocation)
currentConfig.setReportsDir(newTarget)
Unfortunately, although this appears to change the reportsDir in the browser's config object, the actual output still appears in the directory defined by my configuration.
Is this possible? Could I override the setupReporting method in GebReportingTest instead (I've not found anything suggesting how this might be done either)?
--- Edit 1 ---
I've tried
class MyTest extends GebReportingTest {
def pageBrowser
def setup() {
this.pageBrowser = new Browser()
this.pageBrowser.config.reportsDir = new File( 'target/runtime_reports_dir' )
}
#Test
void runTestSet() {
setup()
this.pageBrowser....
}
}
after Tim's comments but I've had no joy so far. After invoking the setup() method, the pageBrowser's config object returns the reportsDir I've defined in code. However, all instances of the 'report' command store screenshots etc in the directory defined in GebConfig.groovy.
Geb will look for GebConfig.groovy in the root of the classpath
You can set reportsDir in there
See: http://www.gebish.org/manual/0.6.2/configuration.html#reports_dir
Edit:
Have you tried:
class MyTest extends GebReportingTest {
void setUp() {
browser.config.reportsDir = new File( 'target/runtime_reports_dir' )
}
#Test
void runTestSet() {
// Do testing
}
}
groovy defualt imports
Is it possible to have our own package in default imports? Is there any way to tell the groovy runtime to use my own package as default import?
This JIRA covers you question.
The bulk of it is here.
class DefaultImportSourceUnitOperation extends SourceUnitOperation {
public void call(SourceUnit source) throws CompilationFailedException {
source.getAST().addImportPackage("pkg1.pgk2.pkg3.");
}
}
class DefaultImportClassLoader extends GroovyClassLoader {
protected CompilationUnit createCompilationUnit(CompilerConfiguration config, CodeSource
codeSource) {
CompilationUnit cu = super.createCompilationUnit(config, codeSource)
cu.addPhaseOperation(new DefaultImportSourceUnitOperation(), Phases.CONVERSION)
return cu
}
}
Be sure not to forget to add the . to the end of the package declaration.
Goodluck!
Since Groovy 1.8 there is an easier way with ImportCustomizer:
def importCustomizer = new ImportCustomizer()
importCustomizer.addImports('com.looneytunes.Coyote', 'com.looneytunes.RoadRunner')
importCustomizer.addStarImports('com.acme')
importCustomizer.addStaticStars('com.looneytunes.Physics')
def configuration = new CompilerConfiguration()
configuration.addCompilationCustomizers(importCustomizer)
def shell = new GroovyShell(configuration)
shell.evaluate """
new Coyote().startPlanToCatch(new RoadRunner()) // no need for explicit imports
.useContraption(new PortableHole()) // from com.acme
.withPhysicsViolation({ forgetGravity(it) }) // method from com.looneytunes.Physics
.start()
"""