I encountered some strange behavior while analyzing the http headers in the following treatment
| sftp folder | ==vfs=> | ESB processing | ===http===> REST endpoint
IN
Files are read from the folder with the following inbound endpoint:
<inboundEndpoint name="aaa2bbb-aaaFolder" onError="aaa2bbb-errors" protocol="file" sequence="aaa2bbb-processFiles" statistics="enable" suspend="false" trace="enable" xmlns="http://ws.apache.org/ns/synapse">
<parameters>
<parameter name="interval">5000</parameter>
<parameter name="sequential">true</parameter>
<parameter name="coordination">true</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.LockReleaseSameNode">false</parameter>
<parameter name="transport.vfs.AutoLockRelease">false</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
<parameter name="transport.vfs.FailedRecordsFileName">vfs-move-failed-records.properties</parameter>
<parameter name="transport.vfs.FailedRecordsFileDestination">repository/conf/</parameter>
<parameter name="transport.vfs.MoveFailedRecordTimestampFormat">dd-MM-yyyy HH:mm:ss</parameter>
<parameter name="transport.vfs.FailedRecordNextRetryDuration">3000</parameter>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter key="gov:/filesystem/fromAAA.txt" name="transport.vfs.FileURI"/>
<parameter name="transport.vfs.ReplyFileURI">${registry:gov:/filesystem/fromAAA.txt}/success</parameter>
<parameter name="transport.vfs.ReplyFileName">response.xml</parameter>
<parameter name="transport.vfs.DistributedLock">false</parameter>
<parameter name="transport.vfs.FileNamePattern">^CMDE.*\.zip|^(?!tmp).*\.xml</parameter>
<parameter name="transport.vfs.Locking">disable</parameter>
<parameter name="transport.vfs.FileSortAttribute">none</parameter>
<parameter name="transport.vfs.FileSortAscending">true</parameter>
<parameter name="transport.vfs.CreateFolder">true</parameter>
<parameter name="transport.vfs.Streaming">false</parameter>
<parameter name="transport.vfs.Build">false</parameter>
</parameters>
</inboundEndpoint>
OUT
The problem is:
vfs variables such as File-Uri, File-Path or Last-Modified systematically end up in the Headers of the requests sent to the REST endpoint
Not only this isn't quite elegant but it raises a real security issue, for sftp File-Uri include username and passwords:
sftp://${user}:${password}#some_domain.com/somePath
Attempts
I tried several variable names to remove this specific header in my sequences:
fileURI
file-uri
File-Uri
FILE-URI
transport.vfs.fileURI
<header name="${some-name}" scope="transport" action="remove"/>
but those variables always end up in the headers. How should I fix it? Is it a matter of tuning the inbound-endpoint properly to encapsulate its transport variables from the ones of the call?
[environment]
wso2ei 6.5.0
fileconnector-2.0.20
Not sure if it helps, but I used the following to remove all transport headers.
<property name="TRANSPORT_HEADERS" action="remove" scope="axis2"/>
Or use the following to remove headers.
<property name="<name of the header to be removed>" scope="transport" action="remove"/>
Maybe its worth a try.
Related
Trying to get NLog logging to the database. Running IIS with .net 7. Have the following packages included in my project
Image of Project Packages
Getting the following error:
Error DatabaseTarget(Name=MessageStoredProcedure): Failed to create ConnectionType from DBProvider=Microsoft.Data.SqlClient.SqlConnection,Microsoft.Data.SqlClient Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Data.SqlClient, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Microsoft.Data.SqlClient, Culture=neutral, PublicKeyToken=null'
My nlog looks like this
<target name="asyncReqResMessage" type="AsyncWrapper">
<target type="Database" name="MessageStoredProcedure">
<keepConnection>true</keepConnection>
<dbProvider>Microsoft.Data.SqlClient.SqlConnection,Microsoft.Data.SqlClient</dbProvider>
<connectionString>Data Source=****.company.org,1433; Initial Catalog=Logging; User ID=*****; Password=****; Connection Timeout=30; Application Name=test;</connectionString>
<commandText>EXEC logInsertExternalRequest #EndPoint, #Method, #LogKey, #Server, #TokenId, #ReceivedDate, #RequestType, #RequestIP, #RequestHeaders, #RequestMessage, #ResponseDate, #ResponseHeaders, #ResponseMessage, #ResponseCode, #ResponseId, #Duration</commandText>
<parameter name="#EndPoint" layout="${event-properties:logInfo:objectpath=EndPoint}" />
<parameter name="#Method" layout="${event-properties:logInfo:objectpath=Method}" />
<parameter name="#LogKey" layout="${event-properties:logInfo:objectpath=LogKey}" />
<parameter name="#Server" layout="${machinename}" />
<parameter name="#TokenId" layout="${event-properties:logInfo:objectpath=TokenId}" />
<parameter name="#ReceivedDate" layout="${event-properties:logInfo:objectpath=ReceivedDate}" />
<parameter name="#RequestIP" layout="${event-properties:logInfo:objectpath=RequestIP}" />
<parameter name="#RequestType" layout="${event-properties:logInfo:objectpath=RequestType}" />
<parameter name="#RequestHeaders" layout="${event-properties:logInfo:objectpath=RequestHeaders}" />
<parameter name="#RequestMessage" layout="${event-properties:logInfo:objectpath=RequestMessage}" />
<parameter name="#ResponseDate" layout="${event-properties:logInfo:objectpath=ResponseDate}" />
<parameter name="#ResponseHeaders" layout="${event-properties:logInfo:objectpath=ResponseHeaders}" />
<parameter name="#ResponseMessage" layout="${event-properties:logInfo:objectpath=ResponseMessage}" />
<parameter name="#ResponseCode" layout="${event-properties:logInfo:objectpath=ResponseCode}" />
<parameter name="#ResponseId" layout="${event-properties:logInfo:objectpath=ResponseId}" />
<parameter name="#Duration" layout="${event-properties:logInfo:objectpath=Duration}" />
</target>
</target>
Made sure the Microsoft.Data.SqlClient.dll is in the bin directory.
Made sure I updated the dbprovder to the following: Microsoft.Data.SqlClient.SqlConnection,Microsoft.Data.SqlClient
Hi I have an XML file like this
<Parameter name="man" override="false" value="false"/>
<Parameter name="vendor" override="false" value="serve"/>
<Parameter name="buzz" override="false" value=""/>
I need to add a line like this
<Parameter name="dentist" override="false" value="sunday"/>
I need to get the file like this along with the space notation
<Parameter name="man" override="false" value="false"/>
<Parameter name="vendor" override="false" value="serve"/>
<Parameter name="dentist" override="false" value="sunday"/>
<Parameter name="buzz" override="false" value=""/>
Thanks in Advance
Using xmlstarlet, the Swiss Army Knife of command-line XML file manipulation (And an actual XML file based on your sample):
$ xmlstarlet ed -a '//Parameter[2]' -t elem -n Parameter \
-i '//Parameter[3]' -t attr -n name -v dentist \
-i '//Parameter[3]' -t attr -n override -v false \
-i '//Parameter[3]' -t attr -n value -v sunday input.xml
<?xml version="1.0"?>
<Parameters>
<Parameter name="man" override="false" value="false"/>
<Parameter name="vendor" override="false" value="serve"/>
<Parameter name="dentist" override="false" value="sunday"/>
<Parameter name="buzz" override="false" value=""/>
</Parameters>
First appends a new Parameter node after the second one (Making it the new third one), and then populates it with attributes.
We're using Cruise Control v1.8.5. We have next scenario:
checkout source code from git repository
run some scripts to build project (e.g. npm install && cordova clean, cordova build, also minify some css files, compile typescript ...)
Now we want to add label on successfull build. Therefore I found that cruisecontrol has labeller option, so I added:
<cb:define name="Mylabeller">
<labeller type="defaultlabeller">
<initialBuildLabel>1</initialBuildLabel>
<labelFormat>0</labelFormat>
<incrementOnFailure>true</incrementOnFailure>
<labelPrefixFile>X:\BuildFiles\myProject\version.txt</labelPrefixFile>
<labelPrefixFileSearchPattern>\d\.\d\.\d\.</labelPrefixFileSearchPattern>
</labeller>
</cb:define>
But the problem is that, this is done before source code is retrieved (git) from repository. I read version from my project and then CruiseControl with regex reads version and outputs something like: 1.0.3.buildNumber.
How to set/update label prefix version after retrieving source code from repository. It reads old one, before checkout. I also read about dynamic variables, can they be somehow set or something?
Appending project's xml:
<cb:scope ProjDirName="myProject">
<project name="myProject" queue="Daily" queuePriority="30">
<workingDirectory>D:\BuildFiles\$(ProjDirName)\Working</workingDirectory>
<artifactDirectory>D:\BuildFiles\$(ProjDirName)\Artifacts</artifactDirectory>
<cb:sourcecontrol module="Modul1" TagOnSuccess="false" Branch="$(BranchDevelop)" />
<tasks>
<cb:NpmInstall />
<cb:Build />
</tasks>
<cb:Mylabeller />
<cb:commonall />
</project>
</cb:scope>
Figured that CruiseControl sets next build version when project build is started and before project's xml is read (checkout, tasks, ...).
CruiseControl project build log:
<parameters>
<parameter name="$CCNetArtifactDirectory" value="D:\BuildFiles\MyProject\Artifacts" />
<parameter name="$CCNetBuildCondition" value="ForceBuild" />
<parameter name="$CCNetBuildDate" value="2016-01-25" />
<parameter name="$CCNetBuildId" value="9d8234720e7342a3aa636aacc120eb13" />
<parameter name="$CCNetBuildTime" value="13:49:42" />
<parameter name="$CCNetFailureTasks" value="System.Collections.ArrayList" />
<parameter name="$CCNetFailureUsers" value="System.Collections.ArrayList" />
<parameter name="$CCNetIntegrationStatus" value="Unknown" />
<parameter name="$CCNetLabel" value="1.0.88" />
<parameter name="$CCNetLastIntegrationStatus" value="Failure" />
<parameter name="$CCNetListenerFile" value="D:\BuildFiles\MyProject\Artifacts\MyProject_ListenFile.xml" />
<parameter name="$CCNetModifyingUsers" value="System.Collections.ArrayList" />
<parameter name="$CCNetNumericLabel" value="88" />
<parameter name="$CCNetProject" value="MyProject" />
<parameter name="$CCNetProjectUrl" value="http://BUILDer/ccnet" />
<parameter name="$CCNetRequestSource" value="machineName1" />
<parameter name="$CCNetUser" value="user1" />
<parameter name="$CCNetWorkingDirectory" value="D:\BuildFiles\MyProject\Working" />
<parameter name="$LastChangeNumber" value="96ece86d55f83c8eb129cbfeb01724a3d37bb18a" />
Also I makes sense that Continuous Integration dictates application version e.g. major.minor.build instead that each commit dictates version.
Solution:
<cb:scope ProjDirName="myProject">
<project name="myProject" queue="Daily" queuePriority="30">
<workingDirectory>D:\BuildFiles\$(ProjDirName)\Working</workingDirectory>
<artifactDirectory>D:\BuildFiles\$(ProjDirName)\Artifacts</artifactDirectory>
<cb:sourceControlMyProject Branch="develop"/>
<cb:dailytrigger />
<tasks>
<cb:NpmInstall />
<cb:BuildRelease />
<cb:PublishAfter />
</tasks>
<publishers>
<cb:commonpublish/>
</publishers>
<cb:MyLabeller LabelPrefix="1.0." /> <!-- version is fixed -->
<cb:commonall />
</project>
</cb:scope>
And labeller:
<cb:define name="MyLabeller">
<labeller type="defaultlabeller">
<initialBuildLabel>1</initialBuildLabel>
<prefix>$(LabelPrefix)</prefix>
<incrementOnFailure>true</incrementOnFailure>
<labelFormat>0</labelFormat>
</labeller>
</cb:define>
Currently this setup is ok. In case we'll be developing version 1.1 upper setup must be changed. Main point was to get versioning working e.g. 1.0. + cruiseControl_next_build_number.
I have a custom SharePoint sandboxed workflow activity that I am trying to pass an optional parameter to. However, ULS gives me the following exception message:
"SJW.SPDActivities.SendEmailWithAttachment.SendEmailWithAttachmentActivity" of type "SendEmailAttachment" failed. The method contains a parameter named "RecipientCC" but the parameters dictionary does not contain a value of this parameter.
My guess is that SPUserCodeWorkflowActionWrapper is using some sort of reflection to bind parameters to my sandboxed workflow method. However, because one of them is declared as optional in my Elements.xml file (RecipientCC), it is unable to find a value to bind and thus throws an exception.
Any idea how to get around this wonky behavior? Do I have to declare all my parameters to be mandatory?
My Elements.xml file:
<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<WorkflowActions>
<Action Name="Send an Email with an Attachment"
SandboxedFunction="true"
Assembly="$SharePoint.Project.AssemblyFullName$"
ClassName="SJW.SPDActivities.SendEmailWithAttachment.SendEmailWithAttachmentActivity"
FunctionName="SendEmailAttachment"
AppliesTo="all"
Category="SJW Custom Actions">
<RuleDesigner Sentence="Email %1 with attachment %2">
<FieldBind Field="RecipientTo,RecipientCC,Subject,Body" Text="these people" Id="1" DesignerType="Email"/>
<FieldBind Field="AttachmentListID,AttachmentListItem" Text="file" Id="2" DesignerType="ChooseDoclibItem"/>
</RuleDesigner>
<Parameters>
<Parameter Name="__Context" Type="Microsoft.SharePoint.WorkflowActions.WorkflowContext, Microsoft.SharePoint.WorkflowActions" Direction="In" DesignerType="Hide"/>
<Parameter Name="RecipientTo" Type="System.Collections.ArrayList, mscorlib" Direction="In"/>
<Parameter Name="RecipientCC" Type="System.Collections.ArrayList, mscorlib" Direction="Optional"/>
<Parameter Name="Subject" Type="System.String, mscorlib" Direction="In"/>
<Parameter Name="Body" Type="System.String, mscorlib" Direction="In"/>
<Parameter Name="AttachmentListID" Type="System.String, mscorlib" Direction="In"/>
<Parameter Name="AttachmentListItem" Type="System.Int32, mscorlib" Direction="In"/>
</Parameters>
</Action>
</WorkflowActions>
</Elements>
My method declaration:
public Hashtable SendEmailAttachment(SPUserCodeWorkflowContext context,
ArrayList RecipientTo,
ArrayList RecipientCC,
string Subject,
string Body,
string AttachmentListID,
int AttachmentListItem)
Try using InitialValue="something" on the Parameter node, restart IIS, remove the workflow association and add it again. Then, on your code, you must ignore the parameter value when it comes as "something"
When I take down the database that backs nlog, nothing gets get logged and it seems NLog swallows the problem. Is there any way to configure it to raise and exception or at least to log in a text file that logging failed?
Here is what my configuration looks like:
<?xml version="1.0" ?>
<nlog autoReload="true" throwExceptions="true" internalLogFile="${basedir}/App_Data/nlog.txt" internalLogLevel="Debug"
internalLogToConsole="true">
<targets>
<!--Useful for debugging-->
<target name="consolelog" type="ColoredConsole"
layout="${date:format=HH\:mm\:ss}|${level}|${stacktrace}|${message}" />
<target name="databaselog" type="Database">
<dbProvider>System.Data.SqlClient</dbProvider>
<!-- database connection parameters -->
<!-- alternatively you could provide a single 'connectionstring' parameter -->
<connectionString>Data Source=.\SQLEXPRESSZ;Initial Catalog=aspnetdb;Integrated Security=SSPI</connectionString>
<commandText>
insert into NLog_Error ([time_stamp],[level],[host],[type],[source],[logger],[message],[stacktrace],[allxml]) values(#time_stamp,#level,#host,#type,#source,#logger,#message,#stacktrace,#allxml);
</commandText>
<parameter name="#time_stamp" layout="${utc_date}" />
<parameter name="#level" layout="${level}" />
<parameter name="#host" layout="${machinename}" />
<parameter name="#type" layout="${exception:format=type}" />
<parameter name="#source" layout="${callsite:className=true:fileName=false:includeSourcePath=false:methodName=false}" />
<parameter name="#logger" layout="${logger}" />
<parameter name="#message" layout="${message}" />
<parameter name="#stacktrace" layout="${exception:stacktrace}" />
<parameter name="#allxml" layout="${web_variables}" />
</target>
</targets>
<rules>
<logger name="*" minlevel="Info" writeTo="databaselog" />
</rules>
</nlog>
You can force Nlog to throw exception when sql server is not reached by following
<nlog throwExceptions="true">
... your nlog config
</nlog>
More info here,
http://nlog-project.org/2010/09/05/new-exception-handling-rules-in-nlog-2-0.html
It's a new feature in v2.0 so you need v2.0.
It will not work in earlier versions.
Also checkout following configuration info
https://github.com/NLog/NLog/wiki/Logging-Troubleshooting
which allows Nlog to log it's own exceptions to a specified file.
Does NLog.config have the property "Copy to Output Directory" set as "Copy always"?
I think you have wrong NLog.config file: you use elements instead of attributes within the target (documentation). Should be something like this:
<target
name="databaselog"
type="Database"
dbProvider="System.Data.SqlClient"
connectionString="Data Source=.\SQLEXPRESSZ;Initial Catalog=aspnetdb;Integrated Security=SSPI"
commandText="insert into NLog_Error ([time_stamp],[level],[host],[type],[source],[logger],[message],[stacktrace],[allxml]) values(#time_stamp,#level,#host,#type,#source,#logger,#message,#stacktrace,#allxml);">
<parameter name="#time_stamp" layout="${utc_date}" />
<parameter name="#level" layout="${level}" />
<parameter name="#host" layout="${machinename}" />
<parameter name="#type" layout="${exception:format=type}" />
<parameter name="#source" layout="${callsite:className=true:fileName=false:includeSourcePath=false:methodName=false}" />
<parameter name="#logger" layout="${logger}" />
<parameter name="#message" layout="${message}" />
<parameter name="#stacktrace" layout="${exception:stacktrace}" />
<parameter name="#allxml" layout="${web_variables}" />
</target>