How to load Resources that are outside of JAR into MultiResourceItemReader? - resources

I'm trying to load Resource[] into a `MultiResourceItemReader. Running the app locally, I can inject the resources directly as they're contained in the resources directory.
However, the app will be deployed as a jar to a server, at which point the resources (flat files) will be located outside of the jar. I'm having trouble using PathMatchingResourcePatternResolver-- I keep getting a "No resources found" error.

MultiResourceItemReader can read from path outside jar. Just try giving absolute path of the resources.
let say you have input files in /data/inputs/csv/ directory with pattern example*.csv, then in application.properties file you can have
input.files.location=/data/inputs/csv/
input.file.pattern=example*.csv
then in the MultiResourceItemReader config, you can give as following,
#Value("file:${input.files.location}${input.file.pattern}")
private Resource[] inputResources; //thus inputResources will have files matching pattern example*.csv in folder
#Bean
public MultiResourceItemReader<T> multiResourceItemReader()
{
MultiResourceItemReader<T> resourceItemReader = new MultiResourceItemReader<T>();
resourceItemReader.setResources(inputResources);
resourceItemReader.setDelegate(reader());
return resourceItemReader;
}

Related

How to add and access a yaml file as a resource in Ballerina?

I have a YAML file that I want to keep as part of my jar resources and consume inside my .bal program.
I have created a resources folder under my project and placed the file there.
The file is being added to the JAR and I can confirm it.
But how do I now access it within my code:
string requestedFile = "./resources/service_openapi.yaml"; // <--- here
res.setFileAsPayload(requestedFile, contentType = mime:APPLICATION_OCTET_STREAM);
The above throws a runtime error: no such file or directory

Can the Virtual File System In Service Stack Be configured not not treat files with multiiple dots as directories?

I'm attempting to serve an angular spa from embedded resources using the ServiceStack Virtual file system.
This appears to be mostly working, however many of my generated files include two dots in the file name like:
inline.bundle.js
I expected that would be served from:
host:port/inline.bundle.js
but it is actually served from
host: port/inline/bundle.js
The virtual filesystem appears to namespace based on the dot. Is there a way to configure it other wise so multiple dots in the last file are not interpretted as directories?
I'm using the angualr-cli to build the app, and it does not expose a way to modify the webpack config generating the bundles, otherwise I would just modify the file names.
The issue is that Embedded Resources don't include paths when the file is embedded so a file in a folder like /inline/bundle.js is embedded with the same resource name that /inline.bundle.js is, i.e: AssemblyNamespace.inline.bundle.js and it's up to ServiceStack's ResourceVirtualDirectory to use common heuristics to predict what the folder is.
You can force the ResourceVirtualPathProvider to treat a file by specifying it in:
SetConfig(new HostConfig {
EmbeddedResourceTreatAsFiles = { "inline.bundle.js" }
});

Could not retrieve information from environment production source(s) file:///

I've got a puppet class defined like this:
class etchostfile
(
$hostfile
)
file { $hostfile :
ensure => file,
source => "file:///var/www/cobbler/pub/hosts-${hostfile}.txt",
path => '/root/hosts',
}
}
Then I've got a node defined:
node 'hostname.fqdn.com'
{
class { 'etchostfile' :
hostfile => foo,
}
}
I want it to take the file /var/www/cobbler/pub/hosts-foo.txt and install it to /root/hosts. But I'm getting this error:
err: /Stage[main]/Etchostfile/File[foo]: Could not evaluate: Could not
retrieve information from environment production source(s)
file:///var/www/cobbler/pub/hosts-foo.txt
The file exists, is readable, and every directory leading to it is at least r-x.
I saw a number of reasons why this error applies to an incorrect puppet:/// source, but I'm using a file:/// source. I also tried disabling SELinux on both agent and master. No luck.
It worked correctly on my test host, so I presume it's a firewall issue. But the agent can get to the master on port 8140, and I already have a signed certificate, and it appears that I am at least getting a catalog, so I don't understand why I can't get a file too.
Looks like you are trying to source the file from your puppet master? In that case, you need to use the puppet:// resource and not the file:// resource.
Also, ensure that your fileserver setup on master is working.
https://docs.puppetlabs.com/guides/file_serving.html#file-server-configuration
[EDIT]
From the above linked doc first paragraph:
If a file resource declaration contains a puppet: URI in its source
attribute, nodes will retrieve that file from the master’s file server
Also, some more doc talking about file source attribute
https://docs.puppetlabs.com/references/latest/type.html#file-attribute-source
If you are trying to reference a local file (local to the node where the agent is running) you can remove the protocol part and just use:
file { $hostfile :
ensure => file,
source => "/var/www/cobbler/pub/hosts-${hostfile}.txt",
path => '/root/hosts',
}
If you are trying to access a file on the puppet master, you need to use the puppet:/// protocol (and not file). This will bring some additional restrictions : usually you do not configure puppet file serving to serve all files on the master (which would be a security issue) but only to serve files that are part of puppet modules.
Or you could use a template with no logic instead of the file:
file { $hostfile :
ensure => file,
content => template("my_module/hosts-${hostfile}.txt.erb"),
path => '/root/hosts',
}
It is counter intuitive, but using templates with no logic tends to have better performance that trying to use Puppet for file serving.

ClassLoader - Loading and saving data

Hopefully someone can help me with this.
It is my understanding that using a ClassLoader is the most reliable way to load in content.
public class Pipeline{
public static URL getResource(String filename) {
return ClassLoader.getSystemResource(filename);
}
public static InputStream getResourceAsStream(String filename) {
return ClassLoader.getSystemResourceAsStream(filename);
}
}
If you had a file at "[jar bundle]/resources/abc.png" ..You would load it by:
URL url = Pipeline.getResource("resources/abc.png");
Loading is simple.
Saving is what's getting me.
I have a program that collects data while running, saves that data on exit, and then loads the data back in next time and keeps adding to it.
Easiest solution I think would be to save back into the jar bundle so that ClassLoader can get at them. Is this even possible? Or recommended?
I don't mind having my resources outside of the jar, just as long as I don't have to resort to 'File' to get at them and save to them. (Unless it can be done cleanly)
folder/application.jar
folder/resources/abc.png
If you could ../ back one from where the ClassLoader is looking it would be easy to cleanly get data from the directory that actually contains the jar file
Pipeline.getResource("../resources/abc.png");
Any ideas?
This isn't really what class loaders are meant for. Loading resources from the class loader is meant so that you can bundle up your application as one package and components can read each other without worrying about how the system you're deploying to is setup.
If the file in the JAR is meant to be changed by the app, then it isn't part of the app and thus probably shouldn't be in the JAR.
I don't have a lot of context on your app, but hopefully my suggestion will be valid for your situation.
I recommend setting a requirement in your app that it has a work area to which it is allowed to read and write and accept a configuration setting that specifies where this directory is. Typical ways to do this in Java are with environment variables, system properties or JNDI settings (for container deployments).
Examples:
Tomcat's startup scripts figure out where it is installed and sets a system property called catalina.home and allows you to over-ride it with an environment variable called CATALINA_HOME.
JBoss looks for JBOSS_HOME
Java application servers typically look for JAVA_HOME to find the JDK.

Uploaded image only available after refreshing the page

When I upload a picture, the file is successfully saved and the path is successfully set. But the uploaded image is not displayed immediately after the form submit. Only when I reload the page, the uploaded image is displayed.
I'm saving the uploaded file as below:
InputStream is;
try {
File file = new File("C:\\****\\*****\\Documents\\NetBeansProjects\\EventsCalendary\\web\\resources\\images\\uploadPhoto.png");
is = event.getFile().getInputstream();
OutputStream os = new FileOutputStream(file);
setUserPhoto("\\EventsCalendary\\resources\\images\\"+file.getName());
byte buf[] = new byte[1024];
int len;
while ((len = is.read(buf)) > 0) {
os.write(buf, 0, len);
}
os.close();
is.close();
} catch (IOException ex) {
System.out.println(ex.getStackTrace());
}
Why is the uploaded image only displayed after reloading the page and how can I solve this?
You're writing the file straight into the IDE's project folder and your intent seems to save the file in the webapp's deploy folder. This is a bad idea and well due to the following 3 main reasons:
Changes in the IDE's project folder does not immediately get reflected in the server's work folder. There's kind of a background job in the IDE which takes care that the server's work folder get synced with last updates (this is in IDE terms called "publishing"). This is the main cause of the problem you're seeing.
In real world code there are circumstances where storing uploaded files in the webapp's deploy folder will not work at all. Some servers do (either by default or by configuration) not expand the deployed WAR file into the local disk file system, but instead fully in the memory. You can't create new files in the memory without basically editing the deployed WAR file and redeploying it.
Even when the server expands the deployed WAR file into the local disk file system, all newly created files will get lost on a redeploy or even a simple restart, simply because those new files are not part of the original WAR file.
You need to write it to a fixed path outside the project/deploy folder instead. For example, /var/webapp/uploads. Then, to get it to be served by your webapp, just add it as a new web application context to the server.
Based on your previous question, I know that you're using Glassfish 3.1. In this server, it's called a "virtual host". You can configure it at server level in the admin console at http://localhost:4848 > Configuration > HTTP Service > Virtual Servers, or at webapp level by adding the following line to the /WEB-INF/glassfish-web.xml (your IDE should have autogenerated one; note that this file is before Glassfish 3.1 called sun-web.xml, so if you're seeing manuals/blogs/tutorials referencing it, yes it's exactly the same file):
<property name="alternatedocroot_1" value="from=/uploads/* dir=/var/webapp" />
Either way, you should then be able to use http://localhost:8080/contextname/uploads/* to serve those uploaded images from by <img> the usual way.
See also:
How to upload files to server using JSP/Servlet?
Recommended way to save uploaded files in a servlet application (contains a Tomcat configuration example)
Reading/writing a text file in a servlet, where should this file be stored in JBoss? (contains JBoss configuration example)
Simplest way to serve static data from outside the application server in a Java web application

Resources