I need some advice on the proper Camel EIP to use to accomplish the following:
I have a Camel ${body} that consists of a List of Maps ...example ( as Groovy code )
def y = [
['sku':'VT-KK2150', 'Performance':'act7', 'ProductGroup':'Jammers', 'name':'ProductPerformance1'],
['sku':'VT-LL2150', 'Performance':'act8', 'ProductGroup':'Sammers', 'name':'ProductPerformance2'],
['sku':'VT-MM2150', 'Performance':'act9', 'ProductGroup':'Bammers', 'name':'ProductPerformance3'],
['sku':'VT-RR2150', 'Performance':'act29', 'ProductGroup':'Bammers', 'name':'ProductPerformance3'] ]
I wish to send a SFTP copy of the Map content to a plain file.
So far, I CAN accomplish the above without trouble. As a matter of clarification, I can see the expected
content of the List by including following in my Camel Route :
<log message="Sku from record 1 = ${body[0]['sku']}"/>
<log message="Performance from record 1 = ${body[0]['Performance']}"/>
<log message="ProductGroup from record 1 = ${body[0]['ProductGroup']}"/>
The problem I have now is that I now need to copy the above content as 3 separate SFTP transmissions
( with 3 different filenames ) based on the value of one of the above Map fields ( in this case the ProductGroup field ).
Example ...in the above example, I would need to send 3 files via SFTP ...one would contain the record for the
Jammers ProductGroup, one would contain the record for the Sammers ProductGroup, and the last one would
contain the 2 records for the Bammers ProductGroup.
Basically, as a first step, I am thinking I need to somehow sort the List based on the ProductGroup field.
I can accomplish that with one line of Groovy code:
y.sort{m1, m2 -> m1.ProductGroup <=> m2.ProductGroup}
The issue now is how to iterate over the Camel ${body} and create a new SFTP transmission when the
ProductGroup name changes.
I am thinking I may be able to do this by implementing some Processor logic and stoing the result
as Header properties ....that said, I am wondering if there is a simpler approach or an existing EIP
for this case.
Related
Need to search multiple product codes using using SolrSearchQueryTemplate so that we can get all products in a single solr call
I am able to search one product code at a time using SolrSearchQueryTemplate but not sure how shall modify solr query so that i can pass multiple product codes in solr request
please find below implementation i have done to search single product code using SolrSearchQueryTemplate and it works fine
$productType=testProductType
$ftsQueryBuilder=multiFieldFreeTextQueryBuilder
INSERT_UPDATE SolrSearchQueryTemplate;name[unique=true];ftsQueryBuilder[default=$ftsQueryBuilder];showFacets;restrictFieldsInResponse;indexedType(Identifier)[default=$productType]
;TESTTEMPLATE;;false;true;
INSERT_UPDATE SolrSearchQueryProperty;ftsQuery;indexedProperty(name)[unique=true];searchQueryTemplate(name)[unique=true]
;true;code;TESTTEMPLATE
final PageableData pageableData = createPageableData(0, getSearchPageSize(), null, ShowMode.Page);
final SearchStateData searchState = new SearchStateData();
final SearchQueryData searchQueryData = new SearchQueryData();
searchQueryData.setValue(XSSFilterUtil.filter(prodIdentifier.get(0)));
searchQueryData.setSearchQueryContext(SearchQueryContext.THIRDPARTYINTEGRATION);
searchState.setQuery(searchQueryData);
pageableData.setFlow(MyConstants.THIRDPARTYINTEGRATION);
final ProductSearchPageData<SearchStateData, de.hybris.platform.commercefacades.product.data.ProductData> searchPageData = testCloudProductSearchFacade
.getProductByCodeMafIdBrand(searchState, pageableData, MyConstants.THIRDPARTYINTEGRATION);
List<ProductData> prodFromSolr = searchPageData.getResults();
let me know what modifications are required to be done in this so that i can pass multiple product codes in request
Note:
searchQueryData.setValue(XSSFilterUtil.filter(prodIdentifier.get(0)));
setValue method accepts single parameter of type String
is it possible to send multiple codes separated by some token which SolrSearchQueryTemplate will automatically consider as multiple search terms
I have 1 database and inside there are maybe say 3 tables, users, user_profile, user_stats. I would like to ask is it possible to have these 3 tables in 1 model file? If user were to visit User A profile, it will show all the User A records from these 3 tables. I can't find it anywhere whether is it possible in CI4. Hope someone here can guide me on this. Thanks in advance guys!
You will need to override the find method. I have done it for findAll and haven't tried for find a method. So you may need to improvise on the following logic. There are three places where based on the condition we are fetching the rows, one check for the array, another for a single row, and last is the default one. So you will need to modify this behavior.
Replace :
$row = $builder
With your custom builder query such as following:
$row = $builder->select("users.*");
$builder->join('user_profile', 'user_profile.id = users.id','inner');
$builder->join('user_stats', 'user_stats.id = users.id','inner');
->whereIn($this->table . '.' . $this->primaryKey, $id)
->get();
You will need to replace the last query according to the conditions in find method and also instead of mentioning users explicitly you should use $this->table and similar for the primary key instead of id, you should use $this->primaryKey.
I have one file which contains some record and i have second file which also contains same record but with more details , so i want to process in a way that read one record from first file and search in second.
How to read two files in parallel ?
Update :
return IntegrationFlows.from(readFilefromDirectory(), fileInboundPollingConsumer())
.split(fileSplitter())
.channel(c ->c.executor(id, executor))
here in first line we are reading first file and then splitting it sending to executer channel so i want to know where should i write logic to read second file means to read from directory and then search record in that excel file .
All the parallel work in Spring Integration can be done via ExecutorChannel.
I'd suggest to use FileSplitter for the first file and an ExecutorChannel as an output.
As for the second file... Well, I'd read it once into the memory e.g. as a Map if you have some delimiter between record and its details. And use that memory store for incoming record.
UPDATE
If the second file is very big, you can do like this:
Scanner scanner=new Scanner(file);
while(scanner.hasNextLine()) {
String line = scanner.nextLine();
if(line.startsWith(payload)) {
return line;
}
}
UPDATE 2
return IntegrationFlows.from(readFilefromDirectory(), fileInboundPollingConsumer())
.split(fileSplitter())
.channel(c ->c.executor(id, executor))
.handle("searchService", "searchMethod")
where searchService is some service #Bean (or just #Service) and searchMethod utilize logic to accept the String line and implement logic to do the search in other file. That search operation will be done in parallel for each splitted like.
The logic to read Excel file is out of Spring Integration scope. That is fully different question.
I'm writing a REST api in node js that will execute a sql query and send the results;
in the request I need to send the WHERE conditions; ex:
GET 127.0.0.1:5007/users //gets the list of users
GET 127.0.0.1:5007/users
id = 1 //gets the user with id 1
Right now the conditions are passed from the client to the rest api in the request's headers.
In the API I'm using sequelize, an ORM that needs to receive WHERE conditions in a particular form (an object); ex: having the condition:
(x=1 AND (y=2 OR z=3)) OR (x=3 AND y=1)
this needs to be formatted as a nested object:
-- x=1
-- AND -| -- y=2
| -- OR ----|
| -- z=3
-- OR -|
|
| -- x=3
-- AND -|
-- y=1
so the object would be:
Sequelize.or (
Sequelize.and (
{x=1},
Sequelize.or(
{y=2},
{z=3}
)
),
Sequelize.and (
{x=3},
{y=1}
)
)
Now I'm trying to pass a simple string (like "(x=1 AND (y=2 OR z=3)) OR (x=3 AND y=1)"), but then I will need a function on the server that can convert the string in the needed object (this method in my opinion has the advantage that the developer writing the client, can pass the where conditions in a simple way, like using sql, and this method is also indipendent from the used ORM, with no need to change the client if we need to change the server or use a different ORM);
The function to read and convert the conditions' string into an object is giving me headache (I'm trying to write one without success, so if you have some examples about how to do something like this...)
What I would like to get is a route capable of executing almost any kind of sql query and give the results:
now I have a different route for everything:
127.0.0.1:5007/users //to get all users
127.0.0.1:5007/users/1 //to get a single user
127.0.0.1:5007/lastusers //to get user registered in the last month
and so on for the other tables i need to query (one route for every kind of request I need in the client);
instead I would like to have only one route, something like:
127.0.0.1:5007/request
(when calling this route I will pass the table name and the conditions' string)
Do you think this solution would be a good solution or you generally use other ways to handle this kind of things?
Do you have any idea on how to write a function to convert the conditions' string into the desired object?
Any suggestion would be appreciated ;)
I would strongly advise you not to expose any part of your database model to your clients. Doing so means you can't change anything you expose without the risk of breaking the clients. One suggestion as far as what you've supplied is that you can and should use query parameters to cut down on the number of endpoints you've got.
GET /users //to get all users
GET /users?registeredInPastDays=30 //to get user registered in the last month
GET /users/1 //to get a single user
Obviously "registeredInPastDays" should be renamed to something less clumsy .. it's just an example.
As far as the conditions string, there ought to be plenty of parsers available online. The grammar looks very straightforward.
IMHO the main disadvantage of your solution is that you are creating just another API for quering data. Why create sthm from scratch if it is already created? You should use existing mature query API and focus on your business logic rather then inventing sthm new.
For example, you can take query syntax from Odata. Many people have been developing that standard for a long time. They have already considered different use cases and obstacles for query API.
Resources are located with a URI. You can use or mix three ways to address them:
Hierarchically with a sequence of path segments:
/users/john/posts/4711
Non hierarchically with query parameters:
/users/john/posts?minVotes=10&minViews=1000&tags=java
With matrix parameters which affect only one path segment:
/users;country=ukraine/posts
This is normally sufficient enough but it has limitations like the maximum length. In your case a problem is that you can't easily describe and and or conjunctions with query parameters. But you can use a custom or standard query syntax. For instance if you want to find all cars or vehicles from Ford except the Capri with a price between $10000 and $20000 Google uses the search parameter
q=cars+OR+vehicles+%22ford%22+-capri+%2410000..%2420000
(the %22 is a escaped ", the %24 a escaped $).
If this does not work for your case and you want to pass data outside of the URI the format is just a matter of your taste. Adding a custom header like X-Filter may be a valid approach. I would tend to use a POST. Although you just want to query data this is still RESTful if you treat your request as the creation of a search result resource:
POST /search HTTP/1.1
your query-data
Your server should return the newly created resource in the Location header:
HTTP/1.1 201 Created
Location: /search/3
The result can still be cached and you can bookmark it or send the link. The downside is that you need an additional POST.
I am using spring integration to download files and to process them.
<int-sftp:inbound-channel-adapter channel="FileDownloadChannel"
session-factory="SftpSessionFactory"
remote-directory="/home/sshaji/from_disney/files"
filter = "modifiedFileListFilter"
local-directory="/home/sshaji/to_disney/downloads"
auto-create-local-directory="true" >
<integration:poller cron="*/10 * * * * *" default="true"/>
</int-sftp:inbound-channel-adapter>
<integration:transformer input-channel="FileDownloadChannel"
ref="ErrorTransformer"
output-channel="EndChannel"/>
<integration:router input-channel="FileErrorProcessingChannel"
expression="payload.getErrorCode() > 0">
<integration:mapping value="true" channel="ReportErrorChannel"/>
<integration:mapping value="false" channel="FilesBackupChannel"/>
</integration:router>
The int-sftp:inbound-channel-adapter is used to download files from sftp server.
It downloads about 6 files. all xml files.
The transformer iterates all the 6 files and check whether they have an error tag.
If there is an error tag then it will set its errorcode as 1. else it will be set a 0.
When it comes out of the transformer and goes to the router,
i want to send the files whose errorcode is set to 1 to move to a specific folder (Error)
and those which has errorcode set to 0 to move to another folder (NoError).
Currently the transformer returns a " list fileNames " which contains the errorcode and fileNames of all the 6 files.
How can i check the error code for each file using the router? and then map that particular file to a router.
Common C Logic for my problem
for (int i =0; i<fileNames.lenght();i++) {
if(fileNames[i].getErrorCode == 1) {
moveToErrorFolder(fileNames[i].getName());
} else {
moveToNoErrors(fileNames[i].getName());
}
}
How can i achieve this using spring integration?.
If its not possible, is there any workaround for it?.
I hope now its clear. I am sorry for not providing enough details last time.
Also in the int-sftp:inbound-channel-adapter i have hard coded the "remote-directory" and "local-directory" fields to a specific folder in the system. can i refer these from a bean property or from a constant value?.
I need to configure these values based on config.xml file, is that possible?.
I am new to Spring Integration. Please help me.
Thanks in Advance.
It's still not clear to me what you mean by "The transformer iterates all the 6 files".
Each file will be passed to the transformer in a single message, so I don't see how it can emit a list of 6.
It sounds like you need an <aggregator/> with a correlation-strategy-expression="'foo'" and release-strategy-expression="size() == 6". This would aggregate each single File into a list of File and pass it to your transformer. It then transforms it to a list of your status objects containing the filename and error code.
Finally, you would add a <splitter/> which would split the list into separate FileName messages to send to the router.
You can use normal Spring property placeholders for the directory attributes ${some.property} or SpEL to use a property of another bean #{someBean.remoteDir}.