Spring Integration Flow Log on Outbound success - spring-integration

how can I add logs if file is transferred successfully.
I want to log file name and some values form my config object
return IntegrationFlows.from(Sftp.inboundAdapter(inboundSftp)
.localDirectory(this.getlocalDirectory(config.getId()))
.deleteRemoteFiles(true)
.autoCreateLocalDirectory(true)
.remoteDirectory(config.getInboundDirectory()), e -> e.poller(Pollers.cron("0 */1 * ? * *").errorChannel(MessageHeaders.ERROR_CHANNEL).errorHandler((ex) -> {
try {
// exception handling here
})))
.handle(Sftp.outboundAdapter(outboundSftp)
.useTemporaryFileName(false)
.autoCreateDirectory(true)
.remoteDirectory(config.getOutboundDirectory()), c -> c.advice(startup.deleteFileAdvice())
)
.get();
Update
after Gary Russell answer, my working code is
return IntegrationFlows.from(Sftp.inboundAdapter(inboundSftp)
.localDirectory(this.getlocalDirectory(config.getId()))
.deleteRemoteFiles(true)
.autoCreateLocalDirectory(true)
.remoteDirectory(config.getInboundDirectory()), e -> e.poller(Pollers.cron("0 */1 * ? * *").errorChannel(MessageHeaders.ERROR_CHANNEL).errorHandler((ex) -> {
// action on exceptions are here
}))).publishSubscribeChannel(s -> s
.subscribe(f -> f
.handle(Sftp.outboundAdapter(outboundSftp)
.useTemporaryFileName(false)
.autoCreateDirectory(true)
.remoteDirectory(config.getOutboundDirectory()), c -> c.advice(startup.deleteFileAdvice())
))
.subscribe(f -> f
.handle(m -> {
// all my custom logging logic is here
})
))
.get();

Add a .publishSubscribeChannel() channel with 2 subflows. Docs here.
.publishSubscribeChannel(s -> s
.subscribe(f -> f
.handle(...)
.subscribe(f -> f
.log())

Related

Spring Integration DSL: handle vs enrichHeaders

I have an integration flow where I use a handler to add a few headers to my message:
...
.handle(this, "enrichTaskHeaders")
...
public Message<?> enrichTaskHeaders(Map<String, String> payload) {
var id = payload.get("id");
var t = service.findById(id);
return MessageBuilder.withPayload(payload).setHeader("a", t.getA())
.setHeader("b", t.getB())
.setHeader(...)
.build();
}
It works perfectly, but since it's about enriching headers, I was wondering if I could use a enrichHeaders method to do the same thing. The closest I could get to is this:
.enrichHeaders(consumer -> consumer.headerFunction("a",
(Message<Map<String, String>> m) -> {
var id = m.getPayload().get("id");
var t = service.findById(id);
return t.getA();})
.headerFunction("b",
(Message<Map<String, String>> m) -> {
var id = m.getPayload().get("id");
var t = service.findById(i));
return t.getB();})
...
)
It works as intended, but it is obviously underefficient since I duplicate a service call for each header I add. Is there a way to rewrite it efficiently or should I just go with the handler?
See enrich(Consumer<EnricherSpec> enricherConfigurer) instead. It has an option like requestPayload() to have that your service call only once. And then you use a .headerFunction() for your purposes.
.enrich(e -> e
.<Map<String, String>>requestPayload(m -> "v")
.headerFunction("a", m -> m.getPayload().getA())
.headerFunction("b", m -> m.getPayload().getB()))
See more in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/message-transformation.html#payload-enricher

wl_display#1.error(wl_display#1, 0, "invalid object 3")

I'm new to wayland, today I wrote a simple wayland server/client sample, but got some error, please help! what I expected is client draw a buffer and commit, then server got this buffer and present to screen by gl/egl.
main part of server:
wl_display_add_socket
....
template <class T>
void DestroyUserData(wl_resource* resource) {
TakeUserDataAs<T>(resource);
}
void compositor_create_surface(wl_client* client,
wl_resource* resource,
uint32_t id) {
wl_resource* surface_resource = wl_resource_create(
client, &wl_surface_interface, wl_resource_get_version(resource), id);
wl_resource_set_implementation(surface_resource, &surface_implementation, nullptr,
DestroyUserData<T>);
}
const struct wl_compositor_interface compositor_implementation = {
compositor_create_surface, compositor_create_region};
const uint32_t compositor_version = 3;
void bind_compositor(wl_client* client,
void* data,
uint32_t version,
uint32_t id) {
wl_resource* resource =
wl_resource_create(client, &wl_compositor_interface,
std::min(version, compositor_version), id);
wl_resource_set_implementation(resource, &compositor_implementation, data,
nullptr);
}
wl_global_create(wl_display_.get(), &wl_compositor_interface,
compositor_version, display_, bind_compositor);
wl_event_loop_dispatch
....
and others
and main part of client:
void Simple::Run(int frames, PresentationFeedback* feedback) {
wl_callback_listener frame_listener = {FrameCallback};
wp_presentation_feedback_listener feedback_listener = {
FeedbackSyncOutput, FeedbackPresented, FeedbackDiscarded};
Presentation presentation;
int frame_count = 0;
std::unique_ptr<wl_callback> frame_callback;
bool frame_callback_pending = false;
do {
if (frame_callback_pending)
continue;
if (frame_count == frames)
break;
Buffer* buffer = DequeueBuffer();
if (!buffer)
continue;
SkCanvas* canvas = buffer->sk_surface->getCanvas();
static const SkColor kColors[] = {SK_ColorRED, SK_ColorBLACK};
canvas->clear(kColors[++frame_count % arraysize(kColors)]);
if (gr_context_) {
gr_context_->flush();
glFinish();
}
wl_surface_set_buffer_scale(surface_.get(), scale_);
wl_surface_set_buffer_transform(surface_.get(), transform_);
wl_surface_damage(surface_.get(), 0, 0, surface_size_.width(),
surface_size_.height());
wl_surface_attach(surface_.get(), buffer->buffer.get(), 0, 0);
// Set up the frame callback.
frame_callback_pending = true;
frame_callback.reset(wl_surface_frame(surface_.get()));
wl_callback_add_listener(frame_callback.get(), &frame_listener,
&frame_callback_pending);
// Set up presentation feedback.
Frame frame;
frame.feedback.reset(
wp_presentation_feedback(globals_.presentation.get(), surface_.get()));
wp_presentation_feedback_add_listener(frame.feedback.get(),
&feedback_listener, &presentation);
frame.submit_time = base::TimeTicks::Now();
presentation.submitted_frames.push_back(std::move(frame));
wl_surface_commit(surface_.get());
wl_display_flush(display_.get());
} while (wl_display_dispatch(display_.get()) != -1);
if (feedback)
*feedback = presentation.feedback;
}
set WAYLAND-DEBUG on, then start server and client,
client report an error, and all the debug info:
[137355.232] -> wl_display#1.get_registry(new id wl_registry#2)
[137355.241] -> wl_display#1.sync(new id wl_callback#3)
[137355.527] wl_display#1.delete_id(3)
[137355.550] wl_registry#2.global(1, "wl_compositor", 3)
[137355.558] -> wl_registry#2.bind(1, "wl_compositor", 3, new id [unknown]#4)
[137355.575] wl_registry#2.global(2, "wl_shm", 1)
[137355.589] -> wl_registry#2.bind(2, "wl_shm", 1, new id [unknown]#5)
[137355.619] wl_registry#2.global(3, "zwp_linux_dmabuf_v1", 2)
[137355.624] -> wl_registry#2.bind(3, "zwp_linux_dmabuf_v1", 1, new id [unknown]#6)
[137355.628] wl_registry#2.global(4, "wl_subcompositor", 1)
[137355.633] -> wl_registry#2.bind(4, "wl_subcompositor", 1, new id [unknown]#7)
[137355.638] wl_registry#2.global(5, "wl_shell", 1)
[137355.675] -> wl_registry#2.bind(5, "wl_shell", 1, new id [unknown]#8)
[137355.697] wl_registry#2.global(6, "wl_output", 2)
[137355.702] wl_registry#2.global(7, "zcr_vsync_feedback_v1", 1)
[137355.726] wl_registry#2.global(8, "wp_presentation", 1)
[137355.732] -> wl_registry#2.bind(8, "wp_presentation", 1, new id [unknown]#9)
[137355.770] wl_registry#2.global(9, "zcr_alpha_compositing_v1", 1)
[137355.781] wl_callback#3.done(0)
[137355.943] -> wl_shm#5.create_pool(new id wl_shm_pool#3, fd 11, 262144)
[137355.983] -> wl_shm_pool#3.create_buffer(new id wl_buffer#10, 0, 256, 256, 1024, 0)
[137356.038] -> wl_shm#5.create_pool(new id wl_shm_pool#11, fd 12, 262144)
[137356.045] -> wl_shm_pool#11.create_buffer(new id wl_buffer#12, 0, 256, 256, 1024, 0)
[137356.081] -> wl_compositor#4.create_surface(new id wl_surface#13)
[137356.096] -> wl_compositor#4.create_region(new id wl_region#14)
[137356.100] -> wl_region#14.add(0, 0, 256, 256)
[137356.106] -> wl_surface#13.set_opaque_region(wl_region#14)
[137356.112] -> wl_region#14.destroy()
[137356.114] -> wl_shell#8.get_shell_surface(new id wl_shell_surface#15, wl_surface#13)
[137356.120] -> wl_shell_surface#15.set_title("Wayland Client")
[137356.125] -> wl_shell_surface#15.set_toplevel()
[137356.425] -> wl_surface#13.set_buffer_scale(1)
[137356.431] -> wl_surface#13.set_buffer_transform(0)
[137356.434] -> wl_surface#13.damage(0, 0, 256, 256)
[137356.455] -> wl_surface#13.attach(wl_buffer#10, 0, 0)
[137356.479] -> wl_surface#13.frame(new id wl_callback#16)
[137356.498] -> wp_presentation#9.feedback(wl_surface#13, new id wp_presentation_feedback#17)
[137356.518] -> wl_surface#13.commit()
[137356.768] wl_display#1.error(wl_display#1, 0, "invalid object 3")
wl_display#1: error 0: invalid object 3
[137356.871] -> wl_shm_pool#11.destroy()
[137356.902] -> wl_buffer#12.destroy()
[137356.913] -> wl_shm_pool#3.destroy()
[137356.957] -> wl_buffer#10.destroy()
[137356.965] -> wl_subcompositor#7.destroy()
[137356.967] -> zwp_linux_dmabuf_v1#6.destroy()
[137356.969] -> wp_presentation#9.destroy()
[137356.972] -> wl_surface#13.destroy()
server log:
[3513445.412] wl_display#1.get_registry(new id wl_registry#2)
[3513445.422] -> wl_registry#2.global(1, "wl_compositor", 3)
[3513445.430] -> wl_registry#2.global(2, "wl_shm", 1)
[3513445.434] -> wl_registry#2.global(3, "zwp_linux_dmabuf_v1", 2)
[3513445.439] -> wl_registry#2.global(4, "wl_subcompositor", 1)
[3513445.442] -> wl_registry#2.global(5, "wl_shell", 1)
[3513445.446] -> wl_registry#2.global(6, "wl_output", 2)
[3513445.450] -> wl_registry#2.global(7, "zcr_vsync_feedback_v1", 1)
[3513445.454] -> wl_registry#2.global(8, "wp_presentation", 1)
[3513445.457] -> wl_registry#2.global(9, "zcr_alpha_compositing_v1", 1)
[3513445.461] wl_display#1.sync(new id wl_callback#3)
[3513445.465] -> wl_callback#3.done(0)
[3513445.467] -> wl_display#1.delete_id(3
[3513446.767] wl_registry#2.bind(1, "wl_compositor", 3, new id [unknown]#4)
[3513446.786] wl_registry#2.bind(2, "wl_shm", 1, new id [unknown]#5)
[3513446.794] -> wl_shm#5.format(875709016)
[3513446.797] -> wl_shm#5.format(875708993)
[3513446.799] -> wl_shm#5.format(1)
[3513446.801] -> wl_shm#5.format(0)
[3513446.803] wl_registry#2.bind(3, "zwp_linux_dmabuf_v1", 1, new id [unknown]#6)
[3513446.808] -> zwp_linux_dmabuf_v1#6.format(909199186)
[3513446.811] -> zwp_linux_dmabuf_v1#6.format(875709016)
[3513446.813] -> zwp_linux_dmabuf_v1#6.format(875708993)
[3513446.815] -> zwp_linux_dmabuf_v1#6.format(875713112)
[3513446.817] -> zwp_linux_dmabuf_v1#6.format(875713089)
[3513446.819] -> zwp_linux_dmabuf_v1#6.format(842094158)
[3513446.821] -> zwp_linux_dmabuf_v1#6.format(842094169)
[3513446.823] wl_registry#2.bind(4, "wl_subcompositor", 1, new id [unknown]#7)
[3513446.829] wl_registry#2.bind(5, "wl_shell", 1, new id [unknown]#8)
[3513446.840] wl_registry#2.bind(8, "wp_presentation", 1, new id [unknown]#9)
[3513446.845] -> wp_presentation#9.clock_id(1)
[3513446.848] wl_shm#5.create_pool(new id wl_shm_pool#3, fd 235, 262144)
[3513446.855] -> wl_display#1.error(wl_display#1, 0, "invalid object 3")

Spring IntegrationFlow CompositeFileListFilter Not Working

I have two filters regexFilter and lastModified.
return IntegrationFlows.from(Sftp.inboundAdapter(inboundSftp)
.localDirectory(this.getlocalDirectory(config.getId()))
.deleteRemoteFiles(true)
.autoCreateLocalDirectory(true)
.regexFilter(config.getRegexFilter())
.filter(new LastModifiedLsEntryFileListFilter())
.remoteDirectory(config.getInboundDirectory())
, e -> e.poller(Pollers.fixedDelay(60_000)
.errorChannel(MessageHeaders.ERROR_CHANNEL).errorHandler((ex) -> {
})))
By googling I understand I have to use CompositeFileListFilter for regex so change my code to
.filter(new CompositeFileListFilter().addFilter(new RegexPatternFileListFilter(config.getRegexFilter())))
Its compiled but on run time throws error and channel stooped and same error goes for
.filter(ftpPersistantFilter(config.getRegexFilter()))
.
.
.
public CompositeFileListFilter ftpPersistantFilter(String regexFilter) {
CompositeFileListFilter filters = new CompositeFileListFilter();
filters.addFilter(new FtpRegexPatternFileListFilter(regexFilter));
return filters;
}
I just want to filter on the basis of file name. There are 2 flows for same remote folder and both are polling with same cron but should pick their relevant file.
EDIT
adding last LastModifiedLsEntryFileListFilter. Its working fine but adding upon request.
public class LastModifiedLsEntryFileListFilter implements FileListFilter<LsEntry> {
private final Logger log = LoggerFactory.getLogger(LastModifiedLsEntryFileListFilter.class);
private static final long DEFAULT_AGE = 60;
private volatile long age = DEFAULT_AGE;
private volatile Map<String, Long> sizeMap = new HashMap<String, Long>();
public long getAge() {
return this.age;
}
public void setAge(long age) {
setAge(age, TimeUnit.SECONDS);
}
public void setAge(long age, TimeUnit unit) {
this.age = unit.toSeconds(age);
}
#Override
public List<LsEntry> filterFiles(LsEntry[] files) {
List<LsEntry> list = new ArrayList<LsEntry>();
long now = System.currentTimeMillis() / 1000;
for (LsEntry file : files) {
if (file.getAttrs()
.isDir()) {
continue;
}
String fileName = file.getFilename();
Long currentSize = file.getAttrs().getSize();
Long oldSize = sizeMap.get(fileName);
if(oldSize == null || currentSize.longValue() != oldSize.longValue() ) {
// putting size in map, will verify in next iteration of scheduler
sizeMap.put(fileName, currentSize);
log.info("[{}] old size [{}] increased to [{}]...", file.getFilename(), oldSize, currentSize);
continue;
}
int lastModifiedTime = file.getAttrs()
.getMTime();
if (lastModifiedTime + this.age <= now ) {
list.add(file);
sizeMap.remove(fileName);
} else {
log.info("File [{}] is still being uploaded...", file.getFilename());
}
}
return list;
}
}
PS : When I am testing filter for regex I have removed LastModifiedLsEntryFileListFilter just for simplicity. So my final Flow is
return IntegrationFlows.from(Sftp.inboundAdapter(inboundSftp)
.localDirectory(this.getlocalDirectory(config.getId()))
.deleteRemoteFiles(true)
.autoCreateLocalDirectory(true)
.filter(new CompositeFileListFilter().addFilter(new RegexPatternFileListFilter(config.getRegexFilter())))
//.filter(new LastModifiedLsEntryFileListFilter())
.remoteDirectory(config.getInboundDirectory()),
e -> e.poller(Pollers.fixedDelay(60_000)
.errorChannel(MessageHeaders.ERROR_CHANNEL).errorHandler((ex) -> {
try {
this.destroy(String.valueOf(config.getId()));
configurationService.removeConfigurationChannelById(config.getId());
// // logging here
} catch (Exception ex1) {
}
}))).publishSubscribeChannel(s -> s
.subscribe(f -> {
f.handle(Sftp.outboundAdapter(outboundSftp)
.useTemporaryFileName(false)
.autoCreateDirectory(true)
.remoteDirectory(config.getOutboundDirectory()), c -> c.advice(startup.deleteFileAdvice()));
})
.subscribe(f -> {
if (doArchive) {
f.handle(Sftp.outboundAdapter(inboundSftp)
.useTemporaryFileName(false)
.autoCreateDirectory(true)
.remoteDirectory(config.getInboundArchiveDirectory()));
} else {
f.handle(m -> {
});
}
})
.subscribe(f -> f
.handle(m -> {
// I am handling exception here
})
))
.get();
and here are exceptions
2020-01-27 21:36:55,731 INFO o.s.i.c.PublishSubscribeChannel - Channel
'application.2.subFlow#0.channel#0' has 0 subscriber(s).
2020-01-27 21:36:55,731 INFO o.s.i.e.EventDrivenConsumer - stopped 2.subFlow#2.org.springframework.integration.config.ConsumerEndpointFactoryBean#0
2020-01-27 21:36:55,731 INFO o.s.i.c.DirectChannel - Channel 'application.2.subFlow#2.channel#0' has 0 subscriber(s).
2020-01-27 21:36:55,731 INFO o.s.i.e.EventDrivenConsumer - stopped 2.subFlow#2.org.springframework.integration.config.ConsumerEndpointFactoryBean#1
EDIT
After passing regex to LastModifiedLsEntryFileListFilter and handle there works for me. When I use any other RegexFilter inside CompositeFileListFilter it thorws error.
.filter(new CompositeFileListFilter().addFilter(new LastModifiedLsEntryFileListFilter(config.getRegexFilter())))
Show, please, your final flow. I don't see that you use LastModifiedLsEntryFileListFilter in your CompositeFileListFilter... You definitely can't use regexFilter() and filter() together - the last one wins. To avoid confusion we suggest to use a filter() and compose all those with CompositeFileListFilter or ChainFileListFilter.
Also what is an error you are mentioning, please.

SequenceType of functions

I have a swift implementation of the haskell <*> operator that seems to work as long as the arguments are arrays:
public func <*> <T, U>(left:[(T)->U], right:[T]) -> [U] {
return flatten(map(left) { (function) -> [U] in
return map(right) { return function($0) }
})
}
I'm trying to make it more general by rewriting it to use sequences instead of arrays, but I'm having difficulty with the syntax to require that a sequence be a sequence of functions. This works to generalize the right argument, but not the left:
public func <*> <T, U, Tseq:SequenceType where Tseq.Generator.Element == T>(left:[(T)->U], right:Tseq) -> [U] {
return flatten(map(left) { (function) -> [U] in
return map(right) { return function($0) }
})
}
Now I'm trying to generalize the left part, but running into syntax errors. It seems like it ought to be:
public func <*> <
T,
U,
Tseq:SequenceType where Tseq.Generator.Element == T,
Fseq:SequenceType where Fseq.Generator.Element == (T) -> U
>(left:[(T)->U], right:Tseq) -> [U] {
return flatten(map(left) { (function) -> [U] in
return map(right) { return function($0) }
})
}
But that gives me an error on the Fseq... line:
Expected '>' to complete generic parameter list
What is the proper syntax (or is there no proper syntax) to require that Fseq.Generator.Element be a function taking a T and returning a U?
Couple of problems (one fixable, one more fatal):
You have the syntax for the generic template a bit off. There’s only one where clause for all the placeholders, rather than an optional one per placeholder. So it would be more like:
public func <*> <
T, U, Tseq: SequenceType, Fseq: SequenceType
where Fseq.enerator.Element == T -> U,
Tseq.Generator.Element == T>
(left:[(T)->U], right:Tseq) -> [U] {
return flatten(map(left) { (function) -> [U] in
return map(right) { return function($0) }
})
}
However, that still won’t work because you can’t have expressions more complex than single types on the rhs of a == (not even tuples of two placeholders). So where Fseq.Generator.Element == T->U is not going to fly.
You might want to look at how swiftz does this – looks like it requires an additional struct.

Force GraphViz force distance between nodes

I use GraphViz with the following dot file:
digraph G
{
rankdir=LR;
subgraph commits
{
"5c071a6b2c" -> "968bda3251" -> "9754d40473" -> "9e59700d33" -> "2a3242efa4";
}
subgraph annotations
{
"V1.0" [shape=box];
"br/HEAD" [shape=box];
"V1.0" -> "9e59700d33" [weight=0];
"br/HEAD" -> "2a3242efa4" [weight=0];
}
}
It give me something like that:
But I want something like that:
V1.0 br/HEAD
| |
\/ \/
5c071a6b2c -> 968bda3251 -> 9754d40473 -> 9e59700d33 -> 2a3242efa4
How can I do that?
For your help,
Thanks by advance.
This will align the annotations with the commits:
digraph G
{
rankdir=LR;
subgraph commits
{
"5c071a6b2c" -> "968bda3251" -> "9754d40473" -> "9e59700d33" -> "2a3242efa4";
}
subgraph annotations1
{
rank="same";
"V1.0" [shape=box];
"V1.0" -> "9e59700d33" [weight=0];
}
subgraph annotations2
{
rank="same";
"br/HEAD" [shape=box];
"br/HEAD" -> "2a3242efa4" [weight=0];
}
}
Since the rank="same"; effects the whole subgraph I had to split the annotations in two different subgraphs.
Result is:

Resources