Has anyone tried to use GridGain as a local cache replacement? With built in eviction and expiration policies, its very attractive.
What is the right way to configure GridGain as a local cache?
EDIT
This is a sample configuration I used to run a simple micro benchmark on the GridGain local cache.
final GridCacheConfiguration cfg = new GridCacheConfiguration();
cfg.setCacheMode(GridCacheMode.LOCAL);
cfg.setSwapEnabled(false);
cfg.setAtomicityMode(GridCacheAtomicityMode.ATOMIC);
cfg.setQueryIndexEnabled(false);
cfg.setBackups(0);
cfg.setStartSize(1000000);
cfg.setName("test");
final GridConfiguration gridConfiguration = new GridConfiguration();
gridConfiguration.setRestEnabled(false);
gridConfiguration.setMarshaller(new GridOptimizedMarshaller());
gridConfiguration.setCacheConfiguration(cfg);
try (final Grid grid = GridGain.start(gridConfiguration)){
final GridCache<String, String> test = grid.cache("test");
final String keyPrefix = "key";
final String valuePrefix = "value";
final LoggingStopWatch stopWatch = new LoggingStopWatch("cacheWrite - GRIDGAIN");
for (int i = 0; i < 1000000; i++) {
test.put(keyPrefix + i, valuePrefix + i);
}
stopWatch.stop();
} catch (GridException e) {
e.printStackTrace();
}
It took around 16 seconds to do 1M synchronous puts (on my Core i7-2640M 2.8GHz laptop). I agree this is too simple a test, but still this is not the performance I was expecting. I was expecting around 1-2 seconds. Do I need to tweak the config to get some more juice out of the cache?
You can definitely configure GridGain as a local cache and take advantage of local transactions, evictions, and expiration policies.
Here is sample spring-based configuration that would do this:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd">
<bean id="grid.cfg" class="org.gridgain.grid.GridConfiguration" scope="singleton">
<property name="cacheConfiguration">
<list>
<bean class="org.gridgain.grid.cache.GridCacheConfiguration">
<property name="name" value="myCache"/>
<property name="cacheMode" value="LOCAL"/>
<!-- Eviction policy. -->
<property name="evictionPolicy">
<bean class="org.gridgain.grid.cache.eviction.lru.GridCacheLruEvictionPolicy">
<property name="maxSize" value="10000"/>
</bean>
</property>
</bean>
</list>
</property>
</bean>
</beans>
You can start the above configuration as following:
$GRIDGAIN_HOME/bin/ggstart.bat path/to/config/file
As far as performance, the issue was fixed in GridGain 6.0.3. The code above executes in less than 1 second for me.
Related
I need an API that will let me download a (e.g. CSV file). I tried to do something like this:
#RequestMapping(value="/{cartId}/export", method = RequestMethod.GET, produces = { "application/octet-stream" })
#ApiOperation(hidden = false, value = "Exports the contents of a cart as a CSV file.", notes = "Exports the contents of a cart as a CSV file.")
#ApiBaseSiteIdUserIdAndCartIdParam
public #ResponseBody ResponseEntity export() {
File file = new File("test.csv");
return ResponseEntity.ok()
.header("Content-Disposition", "attachment; filename=" + "test" + ".csv")
.contentLength(file.length())
.contentType(MediaType.parseMediaType("application/octet-stream"))
.body(new FileSystemResource(file));
}
However, I would get errors like this:
0203_17:48:01,878 INFO [hybrisHTTP21]
[de.hybris.platform.webservicescommons.resolver.RestHandlerExceptionResolver.doResolveException:73]
Translating exception
[org.springframework.web.HttpMediaTypeNotAcceptableException]: Could
not find acceptable representation 0203_17:48:01,879 WARN
[hybrisHTTP21]
[de.hybris.platform.webservicescommons.resolver.AbstractRestHandlerExceptionResolver.writeWithMessageConverters:72]
Could not find HttpMessageConverter that supports return type [class
de.hybris.platform.webservicescommons.dto.error.ErrorListWsDTO] and
[application/octet-stream] 0203_17:48:01,879 WARN [hybrisHTTP21]
[org.springframework.web.servlet.handler.AbstractHandlerExceptionResolver.logException:197]
Resolved [org.springframework.web.HttpMediaTypeNotAcceptableException:
Could not find acceptable representation]
I understand this is because springmvc-servlet.xml only has resolverXStreamJSONConverter and resolverXStreamXmlConverter.
If I were to do it the MVC way, I'd write the file to a HttpServletResponse, but that doesn't seem ideal for OCC. So, how should I implement the file download? (Code sample also appreciated)
NOTE: The API/method will be used with Spartacus storefront.
A workaround is to add org.springframework.http.converter.ResourceHttpMessageConverter in messageConvertersV2 in web/webroot/WEB-INF/config/v2/jaxb-converters-spring.xml:
<util:list id="messageConvertersV2">
<bean class="org.springframework.http.converter.ResourceHttpMessageConverter"/>
<ref bean="customJsonHttpMessageConverter"/>
<ref bean="customXmlHttpMessageConverter"/>
</util:list>
However, not sure if this is correct or a good idea.
I needed a similar implementation to offer a PDF download. The controller now returns ResponseEntity<byte[]> and I appended the corresponding MessageConverter. I also didnt find a 'better' way, but it works.
<bean id="myByteArrayHttpMessageConverter" class="org.springframework.http.converter.ByteArrayHttpMessageConverter"/>
<bean depends-on="messageConvertersV2" parent="listMergeDirective">
<property name="add" ref="myByteArrayHttpMessageConverter"/>
</bean>
I am using int-amqp:outbound-gateway to create a message in Rabbit. The message is published but my flow stops with ReplyRequiredException
setPublisherReturns and setPublisherConfirms are set on the CachingConnectionFactory
when I set requires-reply to 'false' the thread is frozen and doesn't continue
Code:
<rabbit:template id="amqpTemplateCore" connection-factory="connectionFactoryCore" />
<bean id="connectionFactoryCore" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject"><ref bean="rabbitConfiguration" /></property>
<property name="targetMethod"><value>connectionFactory</value></property>
</bean>
<int-amqp:outbound-gateway
request-channel="requestIn"
reply-channel="requestOut"
amqp-template="amqpTemplateCore"
exchange-name="CDS"
routing-key="keyA">
</int-amqp:outbound-gateway>
bean:
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory =
new CachingConnectionFactory(host);
connectionFactory.setVirtualHost(virtualhost);
connectionFactory.setPort(port);
connectionFactory.setUsername(username);
connectionFactory.setPassword(password);
connectionFactory.setPublisherReturns(true);
connectionFactory.setPublisherConfirms(true);
return (ConnectionFactory)connectionFactory;
}
An outbound-gateway is for request/reply scenario. When you send something to an external system and wait for reply from there.
If your logic is one-way, just to send and forget, consider to use <int-amqp:outbound-channel-adapter>.
I have setup the gateway and service, but the web method is not a wrapped object but just 4 Strings.
JAXB raised the error
"javax.xml.bind.MarshalException
- with linked exception:
[com.sun.istack.internal.SAXException2: unable to marshal type "java.lang.String" as an element because it is missing an #XmlRootElement annotation]"
public interface WebMethodGateway {
#Gateway
#Payload("#args[0] + #args[1] + #args[2] + #args[3]")
public Response invoke(String arg1, String arg2, String arg3, String arg4);
}
integration.xml
<int-ws:outbound-gateway id="outboundGateway"
request-channel="requestChannel"
reply-channel="responseChannel"
uri="http://localhost:8080/Service?wsdl"
message-sender="messageSender"
marshaller="marshaller"
unmarshaller="marshaller"/>
<bean id="messageSender" class="org.springframework.ws.transport.http.HttpComponentsMessageSender">
<property name="connectionTimeout" value="5000"/>
<property name="readTimeout" value="10000"/>
</bean>
<int:channel id="requestChannel"/>
<int:channel id="responseChannel"/>
<oxm:jaxb2-marshaller id="marshaller" context-path="myProject.ws.bean" />
<int:gateway id="webMethodGateway"
default-request-channel="requestChannel"
default-reply-channel="responseChannel"
service-interface="myProject.ws.gateway.WebMethodGateway" />
First of all it isn't 4 Strings, it is single concatenated String :
#Payload("#args[0] + #args[1] + #args[2] + #args[3]")
Another concern.
<int-ws:outbound-gateway> is for Web Services, SOAP. Therefore XML.
Right JaxB produces an XML for you, but it does that exactly for domain entities, not simple strings.
You can bypass JaxB with String payload, but that really must be an XML representing <soap:body> content.
Doing exploratory work before I drink the kool-aid.
I am trying to create a simple inbound channel adapter to monitor a directory for new ZIP files.
In order to deal with the ever-present "is it complete?" question, I am trying to adapt the example posted here to incorporate a FileListFilter which checks the modified time of the file.
However, I am getting the following exception:
a boolean result is requiredclass java.util.ArrayList is not assignable to class java.lang.Boolean
at org.springframework.util.Assert.isAssignable(Assert.java:368)
at org.springframework.integration.filter.AbstractMessageProcessingSelector.accept(AbstractMessageProcessingSelector.java:61)
at org.springframework.integration.filter.MessageFilter.handleRequestMessage(MessageFilter.java:103)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:134)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:73)
I had this working well with a simple router based on the file extension, but when I replaced it with this filter it fell apart. Seems the actual list of files is what the Assert is trying to cast to Boolean.
Is it not possible to have a filter wired between an inbound and an outbound adapter? Or must I do the file move to the destination myself in the filter? (The way it is done in the linked example)
Here is the config:
<int-file:inbound-channel-adapter id="filePoller" directory="file:input" channel="filesChannel" filename-pattern="*.zip">
<int:poller fixed-rate="2000" max-messages-per-poll="10" />
</int-file:inbound-channel-adapter>
<int:filter input-channel="filesChannel" ref="lastModifiedFileFilter" output-channel="zipFilesOut"/>
<bean id="lastModifiedFileFilter" class="FileFilterOnLastModifiedTime">
<property name="timeDifference" value="10000"/>
</bean>
<int-file:outbound-channel-adapter id="zipFilesOut" directory="file:target/output/zip" delete-source-files="true" />
Here is the filter:
import java.io.File;
import org.springframework.integration.file.filters.AbstractFileListFilter;
public class FileFilterOnLastModifiedTime extends AbstractFileListFilter<File> {
Long timeDifference = 1000L;
#Override
protected boolean accept(File file) {
long lastModified = file.lastModified();
long currentTime = System.currentTimeMillis();
return (currentTime - lastModified) > timeDifference ;
}
public void setTimeDifference(Long timeDifference) {
this.timeDifference = timeDifference;
}
}
Your FileFilterOnLastModifiedTime bean should be provided to the inbound adapter using the filter attribute.
<int-file:inbound-channel-adapter id="filePoller" directory="file:input" channel="zipFilesOut" filename-pattern="*.zip"
filter="lastModifiedFileFilter">
<int:poller fixed-rate="2000" max-messages-per-poll="10" />
</int-file:inbound-channel-adapter>
An inline <filter/> element is a simple POJO that takes some argument and returns a boolean.
Since you are providing an AbstractFileListFilter, the framework is trying to invoke filterFiles which takes an array and returns a List, not a boolean.
My system has 2 subsystems. Each subsystem has different set of users. Each user has an extra field "SystemName" that can be used to know which system this user belongs to.
In the login forms (1 form for each subsystem) I added a hidden field specifying the type of the form (containing the SystemName value).
Generally, the check is rather simple:
if (user.systemName == params.systemName) {
proceed with regular login
} else {
throw standard login error
}
I tried putting that check to my custom DaoAuthenticationProvider but it has no access to "params.systemName".
Where do I put that code to make Acegi authenticate my users with this check?
Thanks in advance.
This is how I did it in Java. Extend WebAuthenticationDetails:
import javax.servlet.http.HttpServletRequest;
import org.acegisecurity.ui.WebAuthenticationDetails;
public class SystemNameWebAuthenticationDetails extends WebAuthenticationDetails {
public SystemNameWebAuthenticationDetails() {
super();
}
public SystemNameWebAuthenticationDetails(HttpServletRequest request) {
super(request);
this.systemName = request.getParameter("systemName");
}
public String getSystemName() {
return systemName;
}
private String systemName;
}
Set it in the authentication filter:
<bean id="authenticationProcessingFilter"
class="org.acegisecurity.ui.webapp.AuthenticationProcessingFilter">
...
<property name="authenticationDetailsSource">
<bean class="org.acegisecurity.ui.AuthenticationDetailsSourceImpl">
<property name="clazz" value="SystemNameWebAuthenticationDetails"/>
</bean>
</property>
</bean>
Later you can access that property in the authentication process asking the details to the authentication object. Or doing this:
SecurityContextHolder.getContext().getAuthentication().getDetails()