groovy.lang.MissingPropertyException while Downloading Files from FTP Server - groovy

I want to create job in grails which download the files from ftp
server after certain interval of time say 2-3 days and store it on
specified local path. the same code with minor changes is written in
java which was working fine but when write the similar code in Grails
I'm facing the Error and not able to resolve it. Can any body Tell me
where I'm making mistake?
Following is the Error that I'm facing when job start.
JOB STARTED::************************************************************************************
2015-08-24 18:20:35,285 INFO org.quartz.core.JobRunShell:207 Job GRAILS_JOBS.com.hoteligence.connector.job.DownloadIpgGzipFilesJob threw a JobExecutionException:
org.quartz.JobExecutionException: groovy.lang.MissingPropertyException: No such property: ftpClient for class: com.hoteligence.connector.job.DownloadIpgGzipFilesJob [See nested exception: groovy.lang.MissingPropertyException: No such property: ftpClient for class: com.hoteligence.connector.job.DownloadIpgGzipFilesJob]
at grails.plugins.quartz.GrailsJobFactory$GrailsJob.execute(GrailsJobFactory.java:111)
at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
Caused by: groovy.lang.MissingPropertyException: No such property: ftpClient for class: com.hoteligence.connector.job.DownloadIpgGzipFilesJob
at com.hoteligence.connector.job.DownloadIpgGzipFilesJob.execute(DownloadIpgGzipFilesJob.groovy:93)
at grails.plugins.quartz.GrailsJobFactory$GrailsJob.execute(GrailsJobFactory.java:104)
... 2 more
/* I've added all the related dependencies in grails Build Config.
*/
package com.hoteligence.connector.job
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import org.codehaus.groovy.grails.commons.ConfigurationHolder as ConfigHolder;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
import org.apache.commons.net.ftp.FTPReply;
/**
* #author Gajanan
* this is back-end job which download Files from ftp server and store it on locally
*/
class DownloadIpgGzipFilesJob {
static triggers = {
simple repeatInterval: Long.parseLong(ConfigHolder.config.DEVICE_PING_ALERT_JOB_REPEAT_INTERVAL),
startDelay : 60000
}
def execute() {
try{
println "JOB STARTED::************************************************************************************";
/* following is the details which are required for server connectivity
*/
String server = ConfigHolder.config.IPG_SERVER_NAME;
int port = ConfigHolder.config.IPG_SERVER_PORT_NO;
String user = ConfigHolder.config.IPG_SERVER_USER_NAME;
String pass = ConfigHolder.config.IPG_SERVER_USER_PASSWORD;
String [] fileNames = ConfigHolder.config.IPG_DOWNLOADABLE_GZIP_FILE_LIST.split(",");
String downloadFilePath = ConfigHolder.config.IPG_GZIP_DOWNLOAD_LOCATION;
String fileDate = (todaysDate.getYear()+1900)+""+((todaysDate.getMonth()+1)<=9?("0"+(todaysDate.getMonth()+1)):(todaysDate.getMonth()+1))+""+todaysDate.getDate();
FTPClient ftpClient = new FTPClient();
/* Here we are making connection to the server and the reply
from server is printed on console
*/
ftpClient.connect(server, port);
showServerReply(ftpClient);
int replyCode = ftpClient.getReplyCode();
if (!FTPReply.isPositiveCompletion(replyCode)) {
System.out.println("Connect failed");
return;
}
boolean success = ftpClient.login(user, pass);
showServerReply(ftpClient);
if (!success) {
System.out.println("Could not login to the server");
return;
}
/* Here we are iterate the FileList and download them to specified directory
*/
for(int i =0; i<fileNames.length;i++) {
String fileName = "on_"+ConfigHolder.config.IPG_DATA_COUNTRY_CODE+fileNames[i]+fileDate+".xml.gz";
System.out.println(fileName);
downloadFtpFileByName(ftpClient,fileName,downloadFilePath+fileName);
}
}
catch (IOException ex) {
System.out.println("Oops! Something wrong happened");
ex.printStackTrace();
}
catch(Exception e) {
e.printStackTrace();
}
finally {
// logs out and disconnects from server
/* In finally block we forcefully close the connection and close the file node also
*/
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
/* this function is nothing but to print the ftp server reply after connection to ftp server
*/
private static void showServerReply(FTPClient ftpClient) {
String[] replies = ftpClient.getReplyStrings();
if (replies != null && replies.length > 0) {
for (String aReply : replies) {
System.out.println("SERVER: " + aReply);
}
}
}
/* This is the actual logic where we copy the file from ftp
and store on local directory
this method accept three parameter FtpClient object, Name of the file which has to be downloaded from server and the path where downloaded file has to be stored
*/
private static void downloadFtpFileByName(FTPClient ftpClient,String fileName,String downloadfileName){
System.out.println("Strat Time::"+System.currentTimeMillis());
try {
String remoteFile1 = "/"+fileName; // file on server
File downloadFile1 = new File(downloadfileName); // new file which is going to be copied on local directory
OutputStream outputStream1 = new BufferedOutputStream(new FileOutputStream(downloadFile1));
Boolean success = ftpClient.retrieveFile(remoteFile1, outputStream1);
if (success) {
System.out.println("File"+fileName+" has been downloaded successfully.");
}
else
{
System.out.println("File"+fileName+" has been DOWNLOAD FAILURE....");
}
outputStream1.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("END Time::"+System.currentTimeMillis());
}
}

Move this line:
FTPClient ftpClient = new FTPClient();
Outside of the try { ... } catch() block (ie, move it up to before the try)
You are declaring the local variable inside the try, then trying to use it in the finally block

Related

Commons Configuration2 ReloadingFileBasedConfiguration

I am trying to implement the Apache Configuration 2 in my codebase
import java.io.File;
import java.util.concurrent.TimeUnit;
import org.apache.commons.configuration2.PropertiesConfiguration;
import org.apache.commons.configuration2.builder.ConfigurationBuilderEvent;
import org.apache.commons.configuration2.builder.ReloadingFileBasedConfigurationBuilder;
import org.apache.commons.configuration2.builder.fluent.Parameters;
import org.apache.commons.configuration2.convert.DefaultListDelimiterHandler;
import org.apache.commons.configuration2.event.EventListener;
import org.apache.commons.configuration2.ex.ConfigurationException;
import org.apache.commons.configuration2.reloading.PeriodicReloadingTrigger;
import org.apache.commons.configuration2.CompositeConfiguration;
public class Test {
private static final long DELAY_MILLIS = 10 * 60 * 5;
public static void main(String[] args) {
// TODO Auto-generated method stub
CompositeConfiguration compositeConfiguration = new CompositeConfiguration();
PropertiesConfiguration props = null;
try {
props = initPropertiesConfiguration(new File("/tmp/DEV.properties"));
} catch (ConfigurationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
compositeConfiguration.addConfiguration( props );
compositeConfiguration.addEventListener(ConfigurationBuilderEvent.ANY,
new EventListener<ConfigurationBuilderEvent>()
{
#Override
public void onEvent(ConfigurationBuilderEvent event)
{
System.out.println("Event:" + event);
}
});
System.out.println(compositeConfiguration.getString("property1"));
try {
Thread.sleep(14*1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Have a script which changes the value of property1 in DEV.properties
System.out.println(compositeConfiguration.getString("property1"));
}
protected static PropertiesConfiguration initPropertiesConfiguration(File propsFile) throws ConfigurationException {
if(propsFile.exists()) {
final ReloadingFileBasedConfigurationBuilder<PropertiesConfiguration> builder =
new ReloadingFileBasedConfigurationBuilder<PropertiesConfiguration>(PropertiesConfiguration.class)
.configure(new Parameters().fileBased()
.setFile(propsFile)
.setReloadingRefreshDelay(DELAY_MILLIS)
.setThrowExceptionOnMissing(false)
.setListDelimiterHandler(new DefaultListDelimiterHandler(';')));
final PropertiesConfiguration propsConfiguration = builder.getConfiguration();
PeriodicReloadingTrigger trigger = new PeriodicReloadingTrigger(builder.getReloadingController(),
null, 1, TimeUnit.SECONDS);
trigger.start();
return propsConfiguration;
} else {
return new PropertiesConfiguration();
}
}
}
Here is a sample code that I using to check whether the Automatic Reloading works or not. However when the underlying property file is updated, the configuration doesn't reflect it.
As per the documentation :
One important point to keep in mind when using this approach to reloading is that reloads are only functional if the builder is used as central component for accessing configuration data. The configuration instance obtained from the builder will not change automagically! So if an application fetches a configuration object from the builder at startup and then uses it throughout its life time, changes on the external configuration file become never visible. The correct approach is to keep a reference to the builder centrally and obtain the configuration from there every time configuration data is needed.
https://commons.apache.org/proper/commons-configuration/userguide/howto_reloading.html#Reloading_File-based_Configurations
This is different from what the old implementation was.
I was able to successfully execute your sample code by making 2 changes :
make the builder available globally and access the configuration from the builder :
System.out.println(builder.getConfiguration().getString("property1"));
add the listener to the builder :
`builder.addEventListener(ConfigurationBuilderEvent.ANY, new EventListener() {
public void onEvent(ConfigurationBuilderEvent event) {
System.out.println("Event:" + event);
}
});
Posting my sample program, where I was able to successfully demonstrate it
import java.io.File;
import java.util.concurrent.TimeUnit;
import org.apache.commons.configuration2.PropertiesConfiguration;
import org.apache.commons.configuration2.builder.ConfigurationBuilderEvent;
import org.apache.commons.configuration2.builder.ReloadingFileBasedConfigurationBuilder;
import org.apache.commons.configuration2.builder.fluent.Parameters;
import org.apache.commons.configuration2.event.EventListener;
import org.apache.commons.configuration2.reloading.PeriodicReloadingTrigger;
public class TestDynamicProps {
public static void main(String[] args) throws Exception {
Parameters params = new Parameters();
ReloadingFileBasedConfigurationBuilder<PropertiesConfiguration> builder =
new ReloadingFileBasedConfigurationBuilder<PropertiesConfiguration>(PropertiesConfiguration.class)
.configure(params.fileBased()
.setFile(new File("src/main/resources/override.properties")));
PeriodicReloadingTrigger trigger = new PeriodicReloadingTrigger(builder.getReloadingController(),
null, 1, TimeUnit.SECONDS);
trigger.start();
builder.addEventListener(ConfigurationBuilderEvent.ANY, new EventListener<ConfigurationBuilderEvent>() {
public void onEvent(ConfigurationBuilderEvent event) {
System.out.println("Event:" + event);
}
});
while (true) {
Thread.sleep(1000);
System.out.println(builder.getConfiguration().getString("property1"));
}
}
}
The problem with your implementation is, that the reloading is done on the ReloadingFileBasedConfigurationBuilder Object and is not being returned to the PropertiesConfiguration Object.

sphinx-4 NullPointerException at startRecognition

I'm trying to follow this tutorial, and it crashes upon startup after having lots of problems with the dictionary and models, such as.
The dictionary is missing a phonetic transcription for the word 'humphrey'
and
Dec 18, 2014 1:14:50 PM edu.cmu.sphinx.linguist.lextree.HMMTree addPronunciation
SEVERE: Missing HMM for unit T with lc=N rc=EH1
13:14:50.601 SEVERE lexTreeLinguist Bad HMM Unit: EH1
I loaded this dictionary and got the language and acoustic models from their SourceForge page
It then crashes with this:
Exception in thread "main" java.lang.NullPointerException
at edu.cmu.sphinx.linguist.lextree.HMMNode.getBaseUnit(HMMTree.java:506)
at edu.cmu.sphinx.linguist.lextree.HMMNode.<init>(HMMTree.java:484)
at edu.cmu.sphinx.linguist.lextree.Node.addSuccessor(HMMTree.java:165)
at edu.cmu.sphinx.linguist.lextree.HMMTree$EntryPoint.createEntryPointMap(HMMTree.java:1163)
at edu.cmu.sphinx.linguist.lextree.HMMTree$EntryPointTable.createEntryPointMaps(HMMTree.java:1021)
at edu.cmu.sphinx.linguist.lextree.HMMTree.compile(HMMTree.java:795)
at edu.cmu.sphinx.linguist.lextree.HMMTree.<init>(HMMTree.java:716)
at edu.cmu.sphinx.linguist.lextree.LexTreeLinguist.generateHmmTree(LexTreeLinguist.java:433)
at edu.cmu.sphinx.linguist.lextree.LexTreeLinguist.compileGrammar(LexTreeLinguist.java:420)
at edu.cmu.sphinx.linguist.lextree.LexTreeLinguist.allocate(LexTreeLinguist.java:337)
at edu.cmu.sphinx.decoder.search.WordPruningBreadthFirstSearchManager.allocate(WordPruningBreadthFirstSearchManager.java:232)
at edu.cmu.sphinx.decoder.AbstractDecoder.allocate(AbstractDecoder.java:92)
at edu.cmu.sphinx.recognizer.Recognizer.allocate(Recognizer.java:167)
at edu.cmu.sphinx.api.LiveSpeechRecognizer.startRecognition(LiveSpeechRecognizer.java:46)
at com.test.sphinxtest.App.main(App.java:25)
Here's my code.
package com.test.sphinxtest;
import java.io.IOException;
import edu.cmu.sphinx.api.Configuration;
import edu.cmu.sphinx.api.LiveSpeechRecognizer;
import edu.cmu.sphinx.api.SpeechResult;
/**
* Hello world!
*
*/
public class App
{
public static void main( String[] args )
{
Configuration configuration = new Configuration();
configuration.setAcousticModelPath("models/acousticmodel/en-us");
configuration.setDictionaryPath("dictionary/cmudict-0.6d");
configuration.setLanguageModelPath("models/languagemodel/en-us.lm");
try {
LiveSpeechRecognizer recognizer = new LiveSpeechRecognizer(configuration);
recognizer.startRecognition(true);
SpeechResult result = recognizer.getResult();
recognizer.stopRecognition();
while ((result = recognizer.getResult()) != null) {
System.out.println(result.getHypothesis());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The correct dictionary should not have stress marks, you can download it from here:
https://raw.githubusercontent.com/cmusphinx/pocketsphinx/master/model/en-us/cmudict-en-us.dict

Creating a java client for secured esb proxy

I want to create a java client for version proxy service present in wso2 esb. I have secured the version proxy with Username Token Authentication scenario. Now i have started creating the java client to invoke this secured proxy service and my client code is:
package org.wso2.carbon.security.ws;
import org.apache.axis2.addressing.EndpointReference;
import org.apache.axis2.context.ConfigurationContext;
import org.apache.axis2.context.ConfigurationContextFactory;
import org.apache.axis2.description.AxisBinding;
import org.apache.axis2.description.AxisEndpoint;
import org.apache.axis2.rpc.client.RPCServiceClient;
import org.apache.neethi.Policy;
import javax.xml.namespace.QName;
import java.io.BufferedReader;
import java.io.File;
import java.io.InputStreamReader;
import java.net.URL;
import java.util.Map;
public class HelloServiceClient {
static {
System.setProperty("javax.net.ssl.trustStore", "/path/to/keystore" + File.separator+ "wso2carbon.jks");
System.setProperty("javax.net.ssl.trustStorePassword", "wso2carbon");
}
public static void main(String[] args) {
try {
int securityScenario = getSecurityScenario();
String repository = "/path/to/repo" + File.separator + "repository";
ConfigurationContext confContext =
ConfigurationContextFactory.
createConfigurationContextFromFileSystem(repository, null);
String endPoint = "HelloServiceHttpSoap12Endpoint";
if(securityScenario == 1){
endPoint = "HelloServiceHttpsSoap12Endpoint"; // scenario 1 uses HelloServiceHttpsSoap12Endpoint
}
RPCServiceClient dynamicClient =
new RPCServiceClient(confContext,
new URL("http://pc213712:8281/services/Version?wsdl"),
new QName("http://version.services.core.carbon.wso2.org", "Version"),
endPoint);
//Engage Modules
dynamicClient.engageModule("rampart");
dynamicClient.engageModule("addressing");
//TODO : Change the port to monitor the messages through TCPMon
if(securityScenario != 1){
dynamicClient.getOptions().setTo(new EndpointReference("http://pc213712:8281/services/Version/"));
}
//Get the policy from the binding and append the rampartconfig assertion
Map endPoints = dynamicClient.getAxisService().getEndpoints();
AxisBinding axisBinding = ((AxisEndpoint) endPoints.values().iterator().next()).getBinding();
Policy policy = axisBinding.getEffectivePolicy();
**policy.addAssertion(RampartConfigBuilder.createRampartConfig(securityScenario));**
axisBinding.applyPolicy(policy);
//Invoke the service
Object[] returnArray = dynamicClient.invokeBlocking(new QName("http://www.wso2.org/types","greet"),
new Object[]{"Alice"},
new Class[]{String.class});
System.out.println((String) returnArray[0]);
} catch (Exception ex) {
ex.printStackTrace();
}
}
private static int getSecurityScenario() {
int scenarioNumber = 0;
while (scenarioNumber < 1 || scenarioNumber > 15) {
System.out.print("Insert the security scenario no : ");
String inputString = readOption();
try {
scenarioNumber = new Integer(inputString);
} catch (Exception e) {
System.out.println("invalid input, insert a integer between 1 and 15");
}
if(scenarioNumber < 1 || scenarioNumber > 15){
System.out.println("Scenario number should be between 1 and 15");
}
}
return scenarioNumber;
}
private static String readOption() {
try {
BufferedReader console = new BufferedReader(new InputStreamReader(System.in));
String str;
while ((str = console.readLine()).equals("")) {
}
return str;
} catch (Exception e) {
return null;
}
}
}
But in the above code i am struck at one line that is :
policy.addAssertion(RampartConfigBuilder.createRampartConfig(securityScenario));
Here in my rampart_core jar I am getting RampartConfigBuilder class but inside this class there is no such method called createRampartConfig. So i am unable to create Rampart configurations. What can i do to solve this issue? looking forward to your solutions. Thanks in advance

PostSaveDocument call agent asynchronously

I have an Xpage page with a single Notes document datasource.
After saving a document I want to (conditionally) trigger an agent. The agent takes some time to process and we don't want the user to have to wait for the result, so it should be executed asynchronously.
I've managed to get it working from client side JS by using an XHR to the agent URL, but I would like to do it server side so I can control the "Next page" better. When using .run() or .runonserver() the client waits till the agent completes.
Any idea how I could trigger an agent (from SSJS) on PostSaveDocument without the client waiting for the result?
Try to look at Thread and Jobs application on OpenNTF.org. There are nice demos of running task in background, check it here
As Martin suggested I used the JobScheduler example on OpenNtf and modified it to suit my needs. Resulting code can be found below. Any comments or improvements are welcome.
import java.security.AccessController;
import java.security.PrivilegedAction;
import lotus.domino.Agent;
import lotus.domino.Database;
import lotus.domino.NotesException;
import lotus.domino.Session;
import org.eclipse.core.runtime.IProgressMonitor;
import org.eclipse.core.runtime.IStatus;
import org.eclipse.core.runtime.Status;
import org.eclipse.core.runtime.jobs.IJobChangeEvent;
import org.eclipse.core.runtime.jobs.Job;
import org.eclipse.core.runtime.jobs.JobChangeAdapter;
import com.ibm.domino.xsp.module.nsf.ThreadSessionExecutor;
public class JobRunner {
public static void start(String dbPath, String agentName, String paramDocId) {
synchronized (JobRunner.class) {
runningJob = new ISPJob(dbPath, agentName, paramDocId);
runningJob.addJobChangeListener(new JobChangeAdapter() {
public void done(IJobChangeEvent event) {
System.out.println("Done event");
runningJob = null;
}
});
AccessController.doPrivileged(new PrivilegedAction<Object>() {
public Object run() {
runningJob.schedule();
return null;
}
});
}
}
private static ISPJob runningJob;
private static final class ISPJob extends Job {
private ThreadSessionExecutor<IStatus> executor;
private String docId;
private String dbPath;
private String agentName;
public ISPJob(String paramDbPath, String paramAgentName, String paramDocId) {
super(paramDocId);
this.docId = paramDocId;
this.dbPath = paramDbPath;
this.agentName = paramAgentName;
this.executor = new ThreadSessionExecutor<IStatus>() {
#Override
protected IStatus run(Session session) throws NotesException {
System.out.println("Job started" + docId);
System.out.println(" >> Session created: "
+ session.getUserName() + ", Effective User:"
+ session.getEffectiveUserName());
Database db = session.getDatabase(null,dbPath);
if (db != null) {
try {
if (!db.isOpen()) db.open();
if (db.isOpen()) {
System.out.println(" >> Database opened: "
+ db.getTitle());
Agent agent = db.getAgent(agentName);
try {
System.out.println(" >> Agent Started: " + agent.getName());
agent.run(docId);
System.out.println(" >> Agent Ran: " + agent.getName());
} finally {
agent.recycle();
}
}
} finally {
db.recycle();
}
}
System.out.println("Job completed");
return Status.OK_STATUS;
}
};
}
protected IStatus run(IProgressMonitor monitor) {
try {
return executor.run();
} catch (Exception ex) {
return Status.CANCEL_STATUS;
}
}
};
}
You could use a session bean (so it won't get destroyed) that kicks off an Java thread. Or you could issue in code a server console command. Or you implement a DOTS listener.
This may/may not be an option depending on your application requirements but I am having good success calling function in the onClientLoad event which essentially kicks off the process after the XPage has fully loaded.

Why the play(netty3) upload uses single thread?

I use play to develop my project and embedded netty3 as my application server
Please check the following test code:
package controllers;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.commons.io.FileUtils;
import play.Logger;
import play.Play;
import play.mvc.Controller;
import play.mvc.results.RenderText;
public class Upload extends Controller {
private static Integer counter = 0;
private static final Integer MAX = 1;
public static void index() {
render("/upload.html");
}
public static void upload(File file) {
System.out.println("start " + Thread.currentThread());
synchronized (counter) {
System.out.println("middle " + Thread.currentThread());
if (counter > MAX) {
renderText("Sorry, the max upload thread is " + MAX);
} else {
counter++;
uploadFile(file);
counter--;
renderText("Upload success");
}
}
System.out.println("end " + Thread.currentThread());
}
static void uploadFile(File imgFile) {
File file = Play.getFile("/uploads");
try {
FileUtils.copyFileToDirectory(imgFile, file);
} catch (IOException e) {
Logger.error("upload file error", e);
}
}
}
When I opened two browsers(Firefox and Chrome) to upload the files at the same time, I debugged breakpoint in the ' upload(File file)' method. But I found only 1 thread was processing.
After that, then the second request came.
The output is :
start Thread[play-thread-1,5,main]
middle Thread[play-thread-1,5,main]
start Thread[play-thread-1,5,main]
middle Thread[play-thread-1,5,main]
But in Tomcat/Jetty, there were two threads output in the console.
Did any body meet the same problem before ?
I assume that you are running in Dev mode?
The play document says that to make debugging easier, by default Play runs in a single thread model in Dev mode, and in production mode it runs as numbers_of_cores + 1.
You can override this in the application.conf.
# example of using a thread pool of 3
play.pool=3

Resources