I have ExecuteGroovyScript 1.15.3 processor.
Additional classpath property points to java bigquery client and its dependencies.
List of jars:
api-common-1.9.0.jar
auto-value-annotations-1.7.jar
checker-compat-qual-2.5.5.jar
commons-codec-1.11.jar
commons-logging-1.2.jar
error_prone_annotations-2.3.4.jar
failureaccess-1.0.1.jar
gax-1.56.0.jar
gax-httpjson-0.73.0.jar
google-api-client-1.30.9.jar
google-api-services-bigquery-v2-rev20200324-1.30.9.jar
google-auth-library-credentials-0.20.0.jar
google-auth-library-oauth2-http-0.20.0.jar
google-cloud-bigquery-1.111.1.jar
google-cloud-core-1.93.4.jar
google-cloud-core-http-1.93.4.jar
google-http-client-1.34.2.jar
google-http-client-appengine-1.34.2.jar
google-http-client-jackson2-1.34.2.jar
google-oauth-client-1.30.5.jar
grpc-context-1.29.0.jar
gson-2.8.6.jar
guava-29.0-android.jar
httpclient-4.5.11.jar
httpcore-4.4.13.jar
j2objc-annotations-1.3.jar
jackson-core-2.10.2.jar
javax.annotation-api-1.3.2.jar
jsr305-3.0.2.jar
listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
opencensus-api-0.24.0.jar
opencensus-contrib-http-util-0.24.0.jar
proto-google-common-protos-1.17.0.jar
proto-google-iam-v1-0.13.0.jar
protobuf-java-3.11.4.jar
protobuf-java-util-3.11.4.jar
threetenbp-1.4.3.jar
Also processor has CTL.credentials property, value is controller service of GCPCredentialsControllerService type.
Script that instantiates BigQuery:
def creds = CTL.credentials.getGoogleCredentials()
def bigQuery = new BigQueryOptions.DefaultBigQueryFactory().create(
BigQueryOptions.newBuilder()
.setProjectId(ProjectId.evaluateAttributeExpressions().value)
.setCredentials(creds)
.build());
And I got error:
groovy.lang.MissingMethodException: No signature of method:
com.google.cloud.bigquery.BigQueryOptions$Builder.setCredentials() is applicable for argument types:
(com.google.auth.oauth2.ServiceAccountCredentials) values: [ServiceAccountCredentials{....}]
Possible solutions: setCredentials(com.google.auth.Credentials)
This is very strange because ServiceAccountCredentials extends Credentials.
So I tried to explicitly cast it:
def creds = CTL.credentials.getGoogleCredentials() as Credentials
I got error:
org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'ServiceAccountCredentials{....}'
with class 'com.google.auth.oauth2.ServiceAccountCredentials' to class 'com.google.auth.Credentials'
Then I tried to cast to ServiceAccountCredentials and I got:
org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot cast object 'ServiceAccountCredentials{....}'
with class 'com.google.auth.oauth2.ServiceAccountCredentials' to class 'com.google.auth.oauth2.ServiceAccountCredentials'
Why am I getting a GroovyCastException? What could be the problem? Maybe there is an easier way to execute sql?
Related
Summary of the problem:
I'm running a requests call on an API endpoint, whose request params are hidden in a config file and I decided to try out hydra to retrieve those params [Reason being the request params do change as I'm working on collecting custom dataset using RapidAPI]
I have created a class called QueryParamsLocations which implements the getter methods to fetch the parameters to be later used by run_query method.
class QueryParamsLocations(QueryParams):
#hydra.main(config_path='configs', config_name='location_query')
def get_params_query_string(self, cfg: DictConfig) -> dict():
return {
'query': cfg.location_params.query,
'locale': cfg.location_params.locale,
'currency': cfg.location_params.currency
}
#hydra.main(config_path='configs', config_name='location_query')
def get_url(self, cfg: DictConfig) -> str():
return cfg.urls.location_url
#hydra.main(config_path='configs', config_name='location_query')
def get_headers(self, cfg: DictConfig) -> dict():
return {
'X-RapidAPI-Host': cfg.headers.x_rapidapi_host,
'X-RapidAPI-Key': cfg.headers.x_rapidapi_key
}
class QueryParams is an abstract class which has these 3 getter templates. run_query method is an external call to run the request.
#hydra.main(config_path='configs', config_name='location_query')
def run_query(cfg: DictConfig) -> None:
try:
LoggerFactory.get_logger('logs/logger.log', 'INFO').info('Running query for location')
qpl = QueryParamsLocations()
response = requests.request("GET", qpl.get_url(cfg), headers=qpl.get_headers(cfg), params=qpl.get_params_query_string(cfg))
print(response.json())
except Exception as e:
LoggerFactory.get_logger('logs/logger.log',
'ERROR').error(f'Error in running query: {e}')
run_query()
While running run_query without if name == 'main': and with it as well , the following error is encountered :
[2022-05-16 13:43:32,614][logs/logger.log][ERROR] - Error in running query: **decorated_main()** takes from 0 to 1 positional arguments but 2 were given
Although newer version of hydra (I'm using hydra-core==1.1.2) uses two arguments while creating cfg object however , I'm not sure as to whether there's other way of handling this as such.
Also, by searching through other threads, following was also tried - Compose API
however, from the docs, it requires an override parameter , which is not needed atm.
Would like to know if any other approach can be tried out. Happy to provide more details if needed.
Definitely use the Compose API and not hydra.main() for this use case.
You can just pass an empty array for your override list if you have nothing to override.
I extract the Java files using 'wsimport' from https://webservices.netsuite.com/wsdl/v2020_2_0/netsuite.wsdl , but I can't able to bind TokenPassport and others into the Header section.
I create the port object from the NetSuiteService
import org.apache.axis.client.Stub;
import org.apache.axis.message.SOAPHeaderElement;
import com.netsuite.webservices.platform.core_2020_2.GetAllRecord;
import com.netsuite.webservices.platform.core_2020_2.TokenPassport;
import com.netsuite.webservices.platform.core_2020_2.types.GetAllRecordType;
import com.netsuite.webservices.platform_2020_2.NetSuitePortType;
import com.netsuite.webservices.platform_2020_2.NetSuiteService;
NetSuiteService netSuiteService = new NetSuiteService();
NetSuitePortType port= netSuiteService.getNetSuitePort();
'''
Then create TokenPassport and try to bind SOAPHeaderElement to the port using axis Stub; but unfortunately got an exception like class cast exception.
TokenPassport passport = TokenUtil.getPassport();
String namespace = "urn:messages_2020_2.platform.webservices.netsuite.com";
SOAPHeaderElement tokenPassportHeader = new SOAPHeaderElement(namespace,"tokenPassport",passport);
((Stub)port).setHeader(tokenPassportHeader);
java.lang.ClassCastException: com.sun.proxy.$Proxy37 cannot be cast to org.apache.axis.client.Stub
How can i Bind TokenPassport into the Header.?
Based on error you provided,
java.lang.ClassCastException: com.sun.proxy.$Proxy37 cannot be cast to org.apache.axis.client.Stub
it looks like there is no issue with the TokenPassport header but rather runtime class cast exception. Not sure how it happens in your code but there something that bothers me in these two lines:
NetSuiteService netSuiteService = new NetSuiteService();
NetSuitePortType port= netSuiteService.getNetSuitePort();
NetsuiteService is an Interface and requires a dozen of methods to be
implemented so I am not sure how the netSuiteService is instantiated in your code.
to get the instance of NetSuitePortType it is recommended to use the implementation of the NetSuiteServiceLocator interface (which in turn is NetsuiteService)
I have the following models setup:
class Cluster(models.Model):
name = models.CharField(...)
...
class ResourceRequest(Process):
cluster = models.ForeignKey('Cluster')
def clean(self, ...):
if self.cluster.name == 'abc':
...
And when I try to post to:
http://pmas-local:8000/workflow/api/tasks/vm_request/resourcerequest/start/
it complains that ResourceRequest has no cluster.
Stacktrace shows if self.cluster.name == 'abc': caused the problem.
The error is not related to django-viewflow. That's standard django error for a foreign key field, which means that self.cluster is None.
I am trying to get my head around groovy scripting to make some changes to a jenkins pipeline and I keep getting this error:
groovy.lang.MissingPropertyException: No such property: credentials for class:
I have tried declaring the variable with def but I still get the exception, eclipse does not recognise that the property exists.
What am I doing wrong?!
#!/usr/bin/groovy
package common.pipeline
import common.pipeline.Credentials
Credentials credentials = new Credentials()
def withCredentials(steps) {
credentials.productionPipeline(steps)
}
This script will be compiled by groovy into a Script class with the field definition inside the run method, and with another method withCredentials that is trying to access the field (kinda like this):
import common.pipeline.Credentials
class Script1 extends Script {
def withCredentials(steps) {
credentials.productionPipeline(steps)
}
def run(args) {
Credentials credentials = new Credentials()
}
}
As you can see, this won't work, as the credentials aren't at Field level in the class...
Groovy has an annotation to make this happen:
#!/usr/bin/groovy
package common.pipeline
import common.pipeline.Credentials
import groovy.transform.Field
#Field Credentials credentials = new Credentials()
def withCredentials(steps) {
credentials.productionPipeline(steps)
}
In Pentaho Report Designer (PRD), I want to connect to my Data source using a Groovy script (or EMCAScript script) by referring to JNDI created in default.properties file. So, I will create a mult-value list parameter containing JNDI names and connect to the data source based on the JNDI selected in the parameter.
Script:
import groovy.sql.Sql
import javax.naming.InitialContext
import javax.sql.DataSource
class SqlClient {
InitialContext context = new InitialContext()
DataSource dataSource = context.lookup("OLTP") as DataSource
def sql = new Sql(dataSource)
sql.execute("SELECT * FROM Person.User_JNDI")
TypedTableModel model = new TypedTableModel();
while(sql.next())
model.addRow(new Object [] {rs.getString(1)});
c.close();
return model;
}
I am getting the following error:
Caused by: org.apache.bsf.BSFException: exception from Groovy: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
expression: 11: unexpected token: sql # line 11, column 5.
sql.execute("SELECT * FROM Person.User_JNDI")
Also, PRD expects the script to return a model, probably something to do with TypedtableModel class.
Can anyone give me an example of Groovy script?
Assuming that you have filled the default.properties as it says here, you should be able to access the data source with something like:
import groovy.sql.Sql
import javax.naming.InitialContext
import javax.sql.DataSource
class SqlClient {
InitialContext context = new InitialContext()
DataSource dataSource = context.lookup("SampleData") as DataSource
def sql = new Sql(dataSource)
sql.execute '''
select * from ... '''
...
}
Using a Sql instance.