Unable to delete File resources in advice in Spring Integration Flow - spring-integration

I have 3 flows defined in a file, to poll tif files and send it to a channel. The channel is linked to another flow that converts and copies into a pdf file in the same location. Then the third flow ftp's the pdf file. An advice in linked to ftp flow, where both tif and pdf file are to be deleted after successexpression:
#Bean
public IntegrationFlow rtwInflow() {
return IntegrationFlows
.from(rtwTifFileSharePoller()
, e -> e.poller(Pollers.fixedDelay(15000)))
.channel(tifToPdfConverterChannel())
.get();
}
#Bean
public IntegrationFlow rtwTransformFlow() {
return IntegrationFlows
.from(tifToPdfConverterChannel())
.transform(pdfTransfomer)
.log()
.get();
}
#Bean
public IntegrationFlow rtwFtpFlow() {
return IntegrationFlows
.from(rtwPdfFileSharePoller()
, e -> e.poller(Pollers.fixedDelay(15000)))
.handle(ftpOutboundHandler(), out -> out.advice(after()))
.get();
}
The advice looks something like:
#Bean
public ExpressionEvaluatingRequestHandlerAdvice after() {
logger.debug("Evaluating expression advice. ");
ExpressionEvaluatingRequestHandlerAdvice advice = new ExpressionEvaluatingRequestHandlerAdvice();
advice.setOnFailureExpressionString("#root");
advice.setOnSuccessExpressionString("#root");
advice.setSuccessChannel(rtwSourceDeletionChannel());
advice.setFailureChannel(rtwFtpFailureHandleChannel());
advice.setPropagateEvaluationFailures(true);
return advice;
}
The flow upon successful ftp of pdf file, diverts to rtwSourceDeletionChannel(), who does the following:
#Bean
#SuppressWarnings("unchecked")
public IntegrationFlow rtwSourceDeleteAfterFtpFlow() {
return IntegrationFlows
.from(this.rtwSourceDeletionChannel())
.handle(msg -> {
logger.info("Deleting files at source and transformed objects. ");
Message<File> requestedMsg = (Message<File>) msg.getPayload();
String fileName = (String) requestedMsg.getHeaders().get(FileHeaders.FILENAME);
String fileNameWithoutExtn = fileName.substring(0, fileName.lastIndexOf("."));
logger.info("payload: " + msg.getPayload());
logger.info("fileNameWithoutExtn: " + fileNameWithoutExtn);
// delete both pdf and tif files.
File tifFile = new File(rtwSharedPath + File.separator + fileNameWithoutExtn + ".tif");
File pdfFile = new File(rtwSharedPath + File.separator + fileNameWithoutExtn + ".pdf");
while (!tifFile.isDirectory() && tifFile.exists()) {
logger.info("Tif Delete status: " + tifFile.delete());
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
}
while (!pdfFile.isDirectory() && pdfFile.exists()) {
logger.info("PDF Delete status: " + pdfFile.delete());
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
}
}
})
.get();
}
I am getting output as below, where tif file is locked. Using Files.delete() gave me Exception that file is in use by another process.
2019-02-14 21:06:48.882 INFO 972 --- [ask-scheduler-1] nsfomer$$EnhancerBySpringCGLIB$$c667e8e1 : transformed path: \\localhost\atala-capture-upload\45937.pdf
2019-02-14 21:06:48.898 INFO 972 --- [ask-scheduler-1] o.s.integration.handler.LoggingHandler : GenericMessage [payload=145937.pdf, headers={file_originalFile=\\localhost\atala-capture-upload\145937.tif, id=077ad304-efe5-7af5-ed07-17f909f9b0e1, file_name=145937.tif, file_relativePath=145937.tif, timestamp=1550178408898}]
2019-02-14 21:06:53.765 INFO 972 --- [ask-scheduler-2] TWFlows$$EnhancerBySpringCGLIB$$4905989f : Tif Delete status: false
2019-02-14 21:06:58.774 INFO 972 --- [ask-scheduler-2] TWFlows$$EnhancerBySpringCGLIB$$4905989f : Tif Delete status: false
2019-02-14 21:07:03.782 INFO 972 --- [ask-scheduler-2] TWFlows$$EnhancerBySpringCGLIB$$4905989f : Tif Delete status: false
2019-02-14 21:07:08.791 INFO 972 --- [ask-scheduler-2] TWFlows$$EnhancerBySpringCGLIB$$4905989f : Tif Delete status: false
2019-02-14 21:07:13.800 INFO 972 --- [ask-scheduler-2] TWFlows$$EnhancerBySpringCGLIB$$4905989f : Tif Delete status: false
Please help me understand why I am facing this problem. Also, in the pdfTransformer there is no leak, as I have tested the code and I was able to acquire and close FileInputStream on both tif and pdf files.
Also, please guide me if the solution needs to be improved by design.
Thanks in advance....

Well, seems wierd. According to other question on stack overflow
adding System.gc() before delete fixed the problem!

Related

Error while using ServiceBusProcessor - Azure.Messaging.ServiceBus SDK

I am getting an exception after the StartProcessingAsync() method. The debug pointer goes to the "ProcessorErrorAsync" method. I followed similar steps provided in the link - https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/servicebus/Azure.Messaging.ServiceBus/samples/Sample04_Processor.md
Am I missing some steps here?
Exception Details: error.Exception.Message:
Method not found: 'System.Threading.Tasks.Task`1<System.Collections.Generic.IEnumerable`1<Microsoft.Azure.Amqp.AmqpMessage>> Microsoft.Azure.Amqp.ReceivingAmqpLink.ReceiveMessagesAsync(Int32, System.TimeSpan, System.TimeSpan, System.Threading.CancellationToken)'.
private void ListenerBind(string key, ServiceBusProcessorOptions onMessageOptions)
{
ServiceBusClient tempClient = this._cacheClient.Get(key);
ServiceBusProcessor tempProcessor = tempClient.CreateProcessor(this._topicName, this._subscriptionName, onMessageOptions);
try
{
//temp.OnMessageAsync(this.MessageProcessCallBackAsync, onMessageOptions);
tempProcessor.ProcessMessageAsync += MessageProcessCallBackAsync;
tempProcessor.ProcessErrorAsync += ProcessErrorAsync;
tempProcessor.StartProcessingAsync();
}
catch (InvalidOperationException ex)
{
this._logger.Log($"{ex.Message}", EventLevel.Informational);
}
catch (Exception ex)
{
this._logger.LogException(ex);
}
}
private Task ProcessErrorAsync(ProcessErrorEventArgs error)
{
Exception ex = new Exception(
$" , Action {error.ErrorSource}, " +
$" , Endpoint {error.FullyQualifiedNamespace}" +
$",EntityPath {error.EntityPath} "
, error.Exception);
this._logger.LogException(ex);
this._onErrorCallback(ex, string.Empty);
return Task.CompletedTask;
}

File Deleted from MediaStore but not deleted from Storage

I'm creating a Music Player. I know that in Api 29 and above , we should Send Request to User for deleting Music file in MediaStore.
and this is where result get back in Api 29 and above:
intentSenderRequest = registerForActivityResult(new ActivityResultContracts.StartIntentSenderForResult(), new ActivityResultCallback<ActivityResult>() {
#Override
public void onActivityResult(ActivityResult result) {
if (result.getResultCode() == Activity.RESULT_OK) {
if (Build.VERSION.SDK_INT == Build.VERSION_CODES.Q) {
getActivity().getApplication().getContentResolver().delete(deleteUri, null, null);
}
Toast.makeText(getActivity(), deleteAudioModel.getData(), Toast.LENGTH_SHORT).show();
File file = new File(deleteAudioModel.getData());
file.delete();
adapter.itemDeleted(itemForDelete);
} else {
Toast.makeText(getActivity(), "Not Deleted", Toast.LENGTH_SHORT).show();
}
}
});
but the music just delete from MediaStore and remain in device storage.
I tried to delete it after User permission by MediaStore.Audio.Media.DATA and File.delete method,
but I've got no result.
is there any way for delete file in Api 29 and above?

Spring IntegrationFlow CompositeFileListFilter Not Working

I have two filters regexFilter and lastModified.
return IntegrationFlows.from(Sftp.inboundAdapter(inboundSftp)
.localDirectory(this.getlocalDirectory(config.getId()))
.deleteRemoteFiles(true)
.autoCreateLocalDirectory(true)
.regexFilter(config.getRegexFilter())
.filter(new LastModifiedLsEntryFileListFilter())
.remoteDirectory(config.getInboundDirectory())
, e -> e.poller(Pollers.fixedDelay(60_000)
.errorChannel(MessageHeaders.ERROR_CHANNEL).errorHandler((ex) -> {
})))
By googling I understand I have to use CompositeFileListFilter for regex so change my code to
.filter(new CompositeFileListFilter().addFilter(new RegexPatternFileListFilter(config.getRegexFilter())))
Its compiled but on run time throws error and channel stooped and same error goes for
.filter(ftpPersistantFilter(config.getRegexFilter()))
.
.
.
public CompositeFileListFilter ftpPersistantFilter(String regexFilter) {
CompositeFileListFilter filters = new CompositeFileListFilter();
filters.addFilter(new FtpRegexPatternFileListFilter(regexFilter));
return filters;
}
I just want to filter on the basis of file name. There are 2 flows for same remote folder and both are polling with same cron but should pick their relevant file.
EDIT
adding last LastModifiedLsEntryFileListFilter. Its working fine but adding upon request.
public class LastModifiedLsEntryFileListFilter implements FileListFilter<LsEntry> {
private final Logger log = LoggerFactory.getLogger(LastModifiedLsEntryFileListFilter.class);
private static final long DEFAULT_AGE = 60;
private volatile long age = DEFAULT_AGE;
private volatile Map<String, Long> sizeMap = new HashMap<String, Long>();
public long getAge() {
return this.age;
}
public void setAge(long age) {
setAge(age, TimeUnit.SECONDS);
}
public void setAge(long age, TimeUnit unit) {
this.age = unit.toSeconds(age);
}
#Override
public List<LsEntry> filterFiles(LsEntry[] files) {
List<LsEntry> list = new ArrayList<LsEntry>();
long now = System.currentTimeMillis() / 1000;
for (LsEntry file : files) {
if (file.getAttrs()
.isDir()) {
continue;
}
String fileName = file.getFilename();
Long currentSize = file.getAttrs().getSize();
Long oldSize = sizeMap.get(fileName);
if(oldSize == null || currentSize.longValue() != oldSize.longValue() ) {
// putting size in map, will verify in next iteration of scheduler
sizeMap.put(fileName, currentSize);
log.info("[{}] old size [{}] increased to [{}]...", file.getFilename(), oldSize, currentSize);
continue;
}
int lastModifiedTime = file.getAttrs()
.getMTime();
if (lastModifiedTime + this.age <= now ) {
list.add(file);
sizeMap.remove(fileName);
} else {
log.info("File [{}] is still being uploaded...", file.getFilename());
}
}
return list;
}
}
PS : When I am testing filter for regex I have removed LastModifiedLsEntryFileListFilter just for simplicity. So my final Flow is
return IntegrationFlows.from(Sftp.inboundAdapter(inboundSftp)
.localDirectory(this.getlocalDirectory(config.getId()))
.deleteRemoteFiles(true)
.autoCreateLocalDirectory(true)
.filter(new CompositeFileListFilter().addFilter(new RegexPatternFileListFilter(config.getRegexFilter())))
//.filter(new LastModifiedLsEntryFileListFilter())
.remoteDirectory(config.getInboundDirectory()),
e -> e.poller(Pollers.fixedDelay(60_000)
.errorChannel(MessageHeaders.ERROR_CHANNEL).errorHandler((ex) -> {
try {
this.destroy(String.valueOf(config.getId()));
configurationService.removeConfigurationChannelById(config.getId());
// // logging here
} catch (Exception ex1) {
}
}))).publishSubscribeChannel(s -> s
.subscribe(f -> {
f.handle(Sftp.outboundAdapter(outboundSftp)
.useTemporaryFileName(false)
.autoCreateDirectory(true)
.remoteDirectory(config.getOutboundDirectory()), c -> c.advice(startup.deleteFileAdvice()));
})
.subscribe(f -> {
if (doArchive) {
f.handle(Sftp.outboundAdapter(inboundSftp)
.useTemporaryFileName(false)
.autoCreateDirectory(true)
.remoteDirectory(config.getInboundArchiveDirectory()));
} else {
f.handle(m -> {
});
}
})
.subscribe(f -> f
.handle(m -> {
// I am handling exception here
})
))
.get();
and here are exceptions
2020-01-27 21:36:55,731 INFO o.s.i.c.PublishSubscribeChannel - Channel
'application.2.subFlow#0.channel#0' has 0 subscriber(s).
2020-01-27 21:36:55,731 INFO o.s.i.e.EventDrivenConsumer - stopped 2.subFlow#2.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2020-01-27 21:36:55,731 INFO o.s.i.c.DirectChannel - Channel 'application.2.subFlow#2.channel#0' has 0 subscriber(s).
2020-01-27 21:36:55,731 INFO o.s.i.e.EventDrivenConsumer - stopped 2.subFlow#2.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
EDIT
After passing regex to LastModifiedLsEntryFileListFilter and handle there works for me. When I use any other RegexFilter inside CompositeFileListFilter it thorws error.
.filter(new CompositeFileListFilter().addFilter(new LastModifiedLsEntryFileListFilter(config.getRegexFilter())))
Show, please, your final flow. I don't see that you use LastModifiedLsEntryFileListFilter in your CompositeFileListFilter... You definitely can't use regexFilter() and filter() together - the last one wins. To avoid confusion we suggest to use a filter() and compose all those with CompositeFileListFilter or ChainFileListFilter.
Also what is an error you are mentioning, please.

Testing for file upload in Spring MVC

Project setup:
<java.version>1.8</java.version>
<spring.version>4.3.9.RELEASE</spring.version>
<spring.boot.version>1.4.3.RELEASE</spring.boot.version>
We have a REST controller that has a method to upload file like this:
#PostMapping("/spreadsheet/upload")
public ResponseEntity<?> uploadSpreadsheet(#RequestBody MultipartFile file) {
if (null == file || file.isEmpty()) {
return new ResponseEntity<>("please select a file!", HttpStatus.NO_CONTENT);
} else if (blueCostService.isDuplicateSpreadsheetUploaded(file.getOriginalFilename())) {
return new ResponseEntity<>("Duplicate Spreadsheet. Please select a different file to upload",
HttpStatus.CONFLICT);
} else {
try {
saveUploadedFiles(Arrays.asList(file));
} catch (IOException e) {
e.printStackTrace();
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
return new ResponseEntity("Successfully uploaded - " + file.getOriginalFilename(), new HttpHeaders(),
HttpStatus.OK);
}
}
UPDATE:
I've tried this approach from an old example I found, but it doesn't compile cleanly, the MockMvcRequestBuilders.multipart method is not defined....
#Test
public void testUploadSpreadsheet_Empty() throws Exception {
String fileName = "EmptySpreadsheet.xls";
String content = "";
MockMultipartFile mockMultipartFile = new MockMultipartFile(
"emptyFile",
fileName,
"text/plain",
content.getBytes());
System.out.println("emptyFile content is '" + mockMultipartFile.toString() + "'.");
mockMvc.perform(MockMvcRequestBuilders.multipart("/bluecost/spreadsheet/upload")
.file("file", mockMultipartFile.getBytes())
.characterEncoding("UTF-8"))
.andExpect(status().isOk());
}
I believe MockMvcRequestBuilders.multipart() is only available since Spring 5. What you want is MockMvcRequestBuilders.fileUpload() that is available in Spring 4.

Android 6 get path to downloaded file

I our app (Xamarin C#) we download files from a server. At the end of a succeful download we get the URI to the newly-downloaded file and from the URI we get the file path:
Android.Net.Uri uri = downloadManager.GetUriForDownloadedFile(entry.Value);
path = u.EncodedPath;
In Android 4.4.2 and in Android 5 the uri and path look like this:
uri="file:///storage/emulated/0/Download/2.zip"
path = u.EncodedPath ="/storage/emulated/0/Download/2.zip"
We then use path to process the file.
The problem is that in Android 6 (on a real Nexus phone) we get a completely different uri and path:
uri="content://downloads/my_downloads/2802"
path="/my_downloads/2802"
This breaks my code by throwing a FileNotFound exception. Note that the downloaded file exists and is in the Downloads folder.
How can I use the URI I get from Android 6 to get the proper file path so I can to the file and process it?
Thank you,
donescamillo#gmail.com
I didn't get your actual requirement but it looks like you want to process file content. If so it can be done by reading the file content by using file descriptor of downloaded file. Code snippet as
ParcelFileDescriptor parcelFd = null;
try {
parcelFd = mDownloadManager.openDownloadedFile(downloadId);
FileInputStream fileInputStream = new FileInputStream(parcelFd.getFileDescriptor());
} catch (FileNotFoundException e) {
Log.w(TAG, "Error in opening file: " + e.getMessage(), e);
} finally {
if(parcelFd != null) {
try {
parcelFd.close();
} catch (IOException e) {
}
}
}
But I am also looking to move or delete that file after processing.
May you an build your URI with the download folder :
Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS).toURI();
It's work. #2016.6.24
#Override
public void onReceive(Context context, Intent intent) {
String action = intent.getAction();
if(DownloadManager.ACTION_DOWNLOAD_COMPLETE.equals( action)) {
DownloadManager downloadManager = (DownloadManager)context.getSystemService(Context.DOWNLOAD_SERVICE);
long downloadId = intent.getLongExtra(DownloadManager.EXTRA_DOWNLOAD_ID, 0);
DownloadManager.Query query = new DownloadManager.Query();
query.setFilterById(downloadId);
Cursor c = downloadManager.query(query);
if(c != null) {
if (c.moveToFirst()) {
int columnIndex = c.getColumnIndex(DownloadManager.COLUMN_STATUS);
if (DownloadManager.STATUS_SUCCESSFUL == c.getInt(columnIndex)) {
String downloadFileUrl = c.getString(c.getColumnIndex(DownloadManager.COLUMN_LOCAL_URI));
startInstall(context, Uri.parse(downloadFileUrl));
}
}
c.close();
}
}
}
private boolean startInstall(Context context, Uri uri) {
if(!new File( uri.getPath()).exists()) {
System.out.println( " local file has been deleted! ");
return false;
}
Intent intent = new Intent();
intent.addFlags( Intent.FLAG_ACTIVITY_NEW_TASK);
intent.setAction( Intent.ACTION_VIEW);
intent.setDataAndType( uri, "application/vnd.android.package-archive");
context.startActivity( intent);
return true;
}

Resources