I have following configuration for KinesisMessageDrivenChannelAdapter, when I remove dynamoDbMetaDataStore as checkpointer, messages are received correctly, but when I add it back records are always empty.
I debugged the code and KinesisMessageDrivenChannelAdapter.processTask() line 776 (version 2.0.0.M2) returns empty records.
UPDATE:
public DynamoDbMetaDataStore dynamoDbMetaDataStore() {
String url = consumerClientProperties.getDynamoDB().getUrl();
final AmazonDynamoDBAsync amazonDynamoDB = AmazonDynamoDBAsyncClientBuilder.standard()
.withEndpointConfiguration(new EndpointConfiguration(
url,
Regions.fromName(awsRegion).getName()))
.withClientConfiguration(new ClientConfiguration()
.withMaxErrorRetry(consumerClientProperties.getDynamoDB().getRetries())
.withConnectionTimeout(consumerClientProperties.getDynamoDB().getConnectionTimeout())).build();
DynamoDbMetaDataStore dynamoDbMetaDataStore = new DynamoDbMetaDataStore(amazonDynamoDB, "consumer-test");
return dynamoDbMetaDataStore;
}
public KinesisMessageDrivenChannelAdapter kinesisInboundChannel(
AmazonKinesis amazonKinesis, String[] streamNames) {
KinesisMessageDrivenChannelAdapter adapter =
new KinesisMessageDrivenChannelAdapter(amazonKinesis, streamNames);
adapter.setConverter(null);
adapter.setOutputChannel(kinesisReceiveChannel());
adapter.setCheckpointStore(dynamoDbMetaDataStore());
adapter.setConsumerGroup(consumerClientProperties.getName());
adapter.setCheckpointMode(CheckpointMode.manual);
adapter.setListenerMode(ListenerMode.record);
adapter.setStartTimeout(10000);
adapter.setDescribeStreamRetries(1);
adapter.setConcurrency(10);
return adapter;
}
Thank you
I recommend you to test your solution with the latest 2.0.0.BUILD-SNAPSHOT.
There is already an option like:
/**
* Specify a {#link LockRegistry} for an exclusive access to provided streams.
* This is not used when shards-based configuration is provided.
* #param lockRegistry the {#link LockRegistry} to use.
* #since 2.0
*/
public void setLockRegistry(LockRegistry lockRegistry) {
where you would need to inject a DynamoDbLockRegistry for better checkpoint management.
For that purpose you would also need to add this dependency:
compile("com.amazonaws:dynamodb-lock-client:1.0.0")
There indeed might be some issues with filtering in that M2 yet...
Related
Is it possible to have a dynamic reply queue with Jms OutboundGateway via DSL?
Jms.inboundGateway(jmsListenerContainer)
.defaultReplyQueueName("queue1 or queue2")
Working Solution using ThreadLocal and DestinationResolver:
private static final ThreadLocal<String> REPLY_QUEUE = new ThreadLocal<>();
IntegrationFlows.from(Jms.inboundGateway(listenerContainer)
.defaultReplyQueueName("defaultQueue1")
.destinationResolver(destinationResolver())
.transform(p -> {
// on some condition, else "defaultQueue1"
REPLY_QUEUE.set("changedToQueue2");
return p;
})
#Bean
public DestinationResolver destinationResolver() {
return (session, destinationName, pubSubDomain) -> session.createQueue(REPLY_QUEUE.get());
}
It is not clear from where you'd like to take that dynamic reply queue name, but there is another option:
/**
* #param destinationResolver the destinationResolver.
* #return the spec.
* #see ChannelPublishingJmsMessageListener#setDestinationResolver(DestinationResolver)
*/
public S destinationResolver(DestinationResolver destinationResolver) {
By default this one is a DynamicDestinationResolver which does only this: return session.createQueue(queueName);. Probably here you can play somehow with your different names to determine.
Another way is to have a JMSReplyTo property set in the request message from the publisher.
UPDATE
Since you cannot rely on a default Reply-To JMS message property, I suggest you to look into a ThreadLocal in your downstream flow where you can place your custom header. Then a custom DestinationResolver can take a look into that ThreadLocal variable for a name to delegate to the same mentioned DynamicDestinationResolver.
I have written a Java class for event filtering on one of the Publish channels, and rebuilt and deployed it. I have referenced it on the Publish channel too. However, Maximo behaves as if the class was never there.
package com.sof.iface.eventfilter;
import java.rmi.RemoteException;
import psdi.iface.mic.MaximoEventFilter;
import psdi.iface.mic.PublishInfo;
import psdi.mbo.MboRemote;
import psdi.util.MXException;
import psdi.util.logging.MXLogger;
import psdi.util.logging.MXLoggerFactory;
public class VSPPWOCOMPEventFilter extends MaximoEventFilter {
private static final String SILMX_ATTRIBUTE_STATUS = "STATUS";
private MXLogger log = MXLoggerFactory.getLogger("maximo.application.EVENTFILTER");
/**
* Constructor
*
* #param pubInfo Publish Channel Information
* #throws MXException Maximo Exception
*/
public VSPPWOCOMPEventFilter(PublishInfo pubInfo) throws MXException {
super(pubInfo);
} // end constructor.
/**
* Decide whether to filter out the event before it triggers the
* Publish Channel or not.
*/
public boolean filterEvent(MboRemote mbo) throws MXException, RemoteException {
log.debug("######## com.sof.iface.eventfilter.VSPPWOCOMPEventFilter::filterEvent() - Start of Method");
boolean filter = false;
// try {
String status = mbo.getString(SILMX_ATTRIBUTE_STATUS);
log.debug("######## com.sof.iface.eventfilter.VSPPWOCOMPEventFilter::filterEvent() - WO Status " + status);
if(mbo.isModified("STATUS") && status == "COMP") {
log.debug("######## com.sof.iface.eventfilter.VSPPWOCOMPEventFilter::filterEvent() - Skipping MBO");
filter = true;
} else {
filter = super.filterEvent(mbo);
}
log.debug("######## com.sof.iface.eventfilter.VSPPWOCOMPEventFilter::filterEvent() - End of Method");
return filter;
// }
} // end filterEvent.
} // end class.
Please ignore the below text :)
A good logging (tracing) is always a lifesaver when you have problems in a production environment. I will never stop telling to my fellow programmers how much is important to fill code with meaningful log calls.Maximo has a good and flexible logging subsystem. This IBM TechNote describes in detail how logging works in Maximo. Let’s now see hot to use Maximo logging in your custom Java code.
It looks like you need to skip the outbound message when the Work Order is completed. When the event doesn't seem to occur, make sure to check for these flags:
External System is active
Publish Channel is active
Publish Channel listener is enabled
I think you could easily achieve the same result with a SKIP action processing rule. See details here:
https://www.ibm.com/support/knowledgecenter/en/SSLKT6_7.6.0/com.ibm.mt.doc/gp_intfrmwk/c_message_proc_actions.html
Also worth mentioning: IBM added automation script support for Event Filtering in version 7.6 so no more build/redeploy required.
I am trying to write a MediaWiki Search Hook that will list native files in the file system and then, eventually, allow a person to click on one of the files and view its content.
My extensions.json contains this:
"Hooks": {
"SearchAfterNoDirectMatch": "MediaWiki\\Extension\\NativeFileList\\Hooks::onSearchAfterNoDirectMatch"
},
My Hooks::onSearchAfterNoDirectMatch file looks like this:
namespace MediaWiki\Extension\NativeFileList;
class Hooks {
/**
* #see https://www.mediawiki.org/wiki/Manual:Hooks/SearchAfterNoDirectMatch
* #called from https://gerrit.wikimedia.org/g/mediawiki/core/+/master/includes/search/SearchNearMatcher.php
* #param $searchterm
* #param $title - array of titles
* Returns true if it found something, false is otherwise
*/
public static function onSearchAfterNoDirectMatch( $searchterm, &$title ) {
$title=Title::newFromText( "test", "bar");
return false;
}
}
My problem is that no text is returned. Well, it's worse than that. With the above code, I get an exception (but I don't know how to debug it, because I can't see the exception). If I take the line setting $title out, it returns. If i change the line to $title=undefined(); I get another error. If I set $title="foo"; I get no error, but no foo.
So how do I return a search hit or, even better, a set of search hits?
None of the existing search plug-ins use the modern search Hook api, which is documented in these locations:
https://www.mediawiki.org/wiki/Manual:Hooks/SearchAfterNoDirectMatch
https://gerrit.wikimedia.org/g/mediawiki/core/+/master/includes/search/SearchNearMatcher.php
https://doc.wikimedia.org/mediawiki-core/master/php/classSearchNearMatcher.html
That hook can't return text, you just can change the title in order to generate a match from the hook. $title has to be a Title object, if the code you posted above is the exact code you are using your exception is due to the second parameter not being one of the namespace constants like NS_MAIN
SearchAfterNoDirectMatch is used to return the title of a near-match, rather than to supplement the search results. For supplementing search results, use the onSpecialSearchResultsAppend. Here is code adds three lines to the search results:
class Hooks {
/**
* #see https://www.mediawiki.org/wiki/Manual:Hooks/SearchAfterNoDirectMatch
* #called from https://gerrit.wikimedia.org/g/mediawiki/core/+/master/includes/search/SearchNearMatcher.php
* #param $searchterm
* #param $title - array of titles
*/
public static function onSpecialSearchResultsAppend( $that, $out, $term ) {
$out->addHTML("<h3>Extra Search Results:</h3>");
$out->addHTML("<ul>");
$out->addHTML("<li>Extra Result #1</li>");
$out->addHTML("<li>Extra Result #2</li>");
$out->addHTML("<li>Extra Result #3</li>");
$out->addHTML("</ul>");
}
}
}
That should be enough to get most people going.
In Symfony2, I really like the #Security annotation to perform permissions checks.
However, when using this annotation, I come over a limitation which is you can't cumulate them.
/**
* #Security("is_granted('organizer')")
* #Security("is_granted('owner')")
*/
public function displayPlanningsAction($isModel = false)
{
or
* #Security("is_granted('subscription_valid')")
*/
class PlanningController extends CustomBaseController
{
/**
* #Security("is_granted('owner')")
*/
public function displayPlanningsAction($isModel = false)
{
the two previous will only check for is_granted('owner'). That's kind of ok in the first example because I can just put an 'and' and write this in a single line, but it's really annoying for the second example because then I have to repeat for every method of the class.
Is there a simple way I could overcome this (without using the jmsextrasecuritybundle) ?
I get the following error when I try to reach app/example on symfony demo
Error: The Symfony\Component\Security\Core\SecurityContext class is
deprecated since version 2.6 and will be removed in 3.0. Use
Symfony\Component\Security\Core\Authentication\Token\Storage\TokenStorage
or Symfony\Component\Security\Core\Authorization\AuthorizationChecker
instead.
The server is returning the right answer with a 200 status code though.
I've found nothing on Google about it. Has anybody encounter this error before and/or know how to fix it ?
Explanation
Starting with Symfony 2.6 the SecurityContext got split into the TokenStorage and the AuthorizationChecker (see: Symfony Blog - "New in Symfony 2.6: Security component improvements").
The main reason for this was to prevent circular reference which occurred quite often when injecting the SecurityContext into your own services.
Solution
The change itself is 100% backwards compatible (as stated in the linked blog post), you just need to rewrite how you accessed the SecurityContext.
// Symfony 2.5
$user = $this->get('security.context')->getToken()->getUser();
// Symfony 2.6
$user = $this->get('security.token_storage')->getToken()->getUser();
// Symfony 2.5
if (false === $this->get('security.context')->isGranted('ROLE_ADMIN')) { ... }
// Symfony 2.6
if (false === $this->get('security.authorization_checker')->isGranted('ROLE_ADMIN')) { ... }
You can simply try to find the culprit by doing a text-search for security.context or SecurityContext in your source code (including the vendor directory).
But as you stated that you're using vanilla Symfony 2.6 it seems that it simply uses some soon to be deprecated methods. So you might simply use this...
Workaround
As Symfony does it's deprecation by triggering E_USER_DEPRECATED errors, you can simply disable them when booting your Symfony AppKernel:
// app/AppKernel.php
class AppKernel extends Kernel
{
public function __construct($environment, $debug) {
// Keep error reporting like it was and disable only deprecation warnings.
error_reporting(error_reporting() & (-1 ^ E_DEPRECATED));
// ...
}
}
I personally like the deprecation warnings, because Symfony's changelogs tend to give very detailed information on how you need to change your code to support future versions of Symfony and the deprecation warnings normally are triggered months before the methods are actually deprecated.
It's not a proper error, just a warning.
A deprecated class is a class that is planned to be removed in future releases (of Symfony, in this case).
It suggest you to stop using it, and points you to the newer (and substitutes) class, TokenStorage and AuthorizationChecker, that will take completly over to do the same tasks.
It gets so annoying to see that warning. At the same time you don't want to turn off the warnings. So I thought maybe it's useful to give an example of changing your code to get rid of it. Here's how I changed HWIOAuthBundle's OAuthUtils class to do so.
First, I changed /vendor/hwi/oauth-bundle/HWI/Bundle/OAuthBundle/Resources/config/oauth.html from this:
<service id="hwi_oauth.security.oauth_utils" class="%hwi_oauth.security.oauth_utils.class%">
<argument type="service" id="security.http_utils" />
<argument type="service" id="security.context" />
<argument>%hwi_oauth.connect%</argument>
</service>
to this:
<service id="hwi_oauth.security.oauth_utils" class="%hwi_oauth.security.oauth_utils.class%">
<argument type="service" id="security.http_utils" />
<argument type="service" id="security.authorization_checker" />
<argument>%hwi_oauth.connect%</argument>
</service>
Now we have to change it in the /vendor/hwi/oauth-bundle/HWI/Bundle/OAuthBundle/Security/OAuthUtils class from this:
use Symfony\Component\Security\Core\SecurityContextInterface;
...
/**
* #var SecurityContextInterface
*/
private $securityContext;
/**
* #param HttpUtils $httpUtils
* #param SecurityContextInterface $securityContext
* #param boolean $connect
*/
public function __construct(HttpUtils $httpUtils, SecurityContextInterface $securityContext, $connect)
{
$this->httpUtils = $httpUtils;
$this->securityContext = $securityContext;
$this->connect = $connect;
}
to this:
use Symfony\Component\Security\Core\Authorization\AuthorizationChecker;
...
/**
* #var AuthorizationChecker
*/
private $authorizationChecker;
/**
* #param HttpUtils $httpUtils
* #param AuthorizationChecker $authorizationChecker
* #param boolean $connect
*/
public function __construct(HttpUtils $httpUtils, AuthorizationChecker $authorizationChecker, $connect)
{
$this->httpUtils = $httpUtils;
$this->authorizationChecker = $authorizationChecker;
$this->connect = $connect;
}
Then I made changes where the securityContext was used. Replaced it with authorizationChecker.
public function getAuthorizationUrl(Request $request, $name, $redirectUrl = null, array $extraParameters = array())
{
$resourceOwner = $this->getResourceOwner($name);
if (null === $redirectUrl) {
if (!$this->connect || !$this->authorizationChecker->isGranted('IS_AUTHENTICATED_REMEMBERED')) {
$redirectUrl = $this->httpUtils->generateUri($request, $this->ownerMap->getResourceOwnerCheckPath($name));
} else {
$redirectUrl = $this->getServiceAuthUrl($request, $resourceOwner);
}
}
return $resourceOwner->getAuthorizationUrl($redirectUrl, $extraParameters);
}
The reason of replacing SecurityContext with AuthorizationChecker is because only isGranted method is used in this case. Maybe you could replace it with TokenStorage or use both AuthorizationChecker and TokenStorage if you needed for your case.