I have set up a logic app with the ftp trigger [When a file is added or modified (properties only)]. This works just fine when I upload a 50+MB file to that ftp server. I have a [Get File Content] action set up right after the trigger. For the File input of the [Get File Content] action, I used the [List of Files Name] dynamic content from the trigger AND I have just filled in the path by using the available 'File Picker' (which connects to the FTP just fine). When I test this out, it fails on the [Get File Content] action stating BadRequest and this Body.
{
"status": 400,
"message": "An invalid request was made. Inspect the passed parameters and actions.\r\nclientRequestId: 7d9f2ff3-62d0-4f69-8cc5-f41c35297882",
"source": "ftp-eus.azconn-eus.p.azurewebsites.net"
}
The inputs that come into the action show the correct file name and path. So I am confused on what it means by "Inspect the passed parameters and actions". Can someone point me in the right direction on how to solve this?
EDIT
Here are some screenshots to show. I don't get [File Name] as a dynamic option from my trigger. It doesn't even matter though, I can pick the exact file I want downloaded from the FTP Picker and it still fails. See screenshots:
Dynamically select file:
Statically select file:
Same result from both of them:
If you use Get file content to pick file, you could find the input of File is the Path of the file you want, so you could not get the file content just with the File content. You could use File path or File name, if you want to use the File name you should also know the path.
If you want to use File name, the input would be like this, this is a little inconvenient.
Or just with the File path. Actually the inputs in these two ways are same, so they all could get the file.
Related
I got a sftp location where generally .csv files are put, and we process or pull the file present there using ADF Copy activity. When no file is present and we give the exact filename and run the pipeline, it fails which is as expected. But when we give a wildcard character such as abc*.csv and run the pipeline with no file present in the sftp location, the copy activity passes though rows written is 0. Can anyone tell me why this happens. We are using Adf v2.
The answer to this is nuanced. Here is the difference:
When you give an exact filename, but the file doesn't exist, Data Factory tries to get it, but the request is returned a 'file not found' error. This is passed up to the activity, and is recognized as a failure.
When you give a wildcard, this is really asking "Get me a list of files that fit this pattern, and then Copy each of them". When there are no files matching the pattern, the result is an empty list. Since the list length is 0, no requests to fetch any file is made, meaning there is no opportunity to be served a 'file not found' error.
This is my reasoning from my experience with Data Factory. I am not a member of the development team.
I wonder if you can configure logstash in the following way:
Background Info:
Every day I get a xml file pushed to my server, which should be parsed.
To indicate a complete file transfer afterwards I get an empty .ctl (custom file) transfered to the same folder.
The files both have the following name schema 'feedback_{year}{yearday}_UTC{hoursminutesseconds}_51.{extention}' (e.g. feedback_16002_UTC235953_51.xml). So they have the same file name but one is with .xml and the other is a .ctl file.
Question:
Is there a way to configure logstash to wait parsing the xml file until the according .ctl file is present?
EDIT:
Is there maybe a way to archiev that with filebeat?
EDIT2:
It would also be enough to be able to configure logstash in a way that it will wait x minutes before starting to process a new file, if that is easier.
Thanks for any help in advance
Your problem is that you don't want to start the parser before the file transfer hasn't been completed. So, why don't push the data to a file (file-complete.xml) when you find your flag file (empty.ctl)?
Here is the possible logic for a script and runs using crontab:
if empty.ctl exists:
Clear file-complete.xml
Add the content of file.xml to file-complete.xml.
Remove empty.ctl
This way, you'd need to parse the data from file-complete.xml. I think is simpler to debug and configure.
Hope it helps,
Does anyone know how to get the full path of a DirectoryEntry object in a Chrome Packaged App, without any tildes or other shortcuts?
I am writing a Google Chrome Packaged App. My app has a button where a user can choose a directory using chrome.fileSystem API. When the directory choice comes back to my app, it is represented by a DirectoryEntry object, which is defined in the File API. The object looks like this in the console:
DirectoryEntry {
filesystem: DOMFileSystem
fullPath: "/to_read"
isDirectory: true
isFile: false
name: "to_read"
__proto__: DirectoryEntry
}
I am using Windows and the full path of the directory is
C:\Users\David\Desktop\to_read
I would like a function that can return that path or something close. Unfortunately, the closest thing I found is chrome.fileSystem.getDisplayPath, but that returns the following:
~\Desktop\to_read
The return value from getDisplayPath is not useful to me, because I want to get the full name of the directory (including the drive) so I can compare it to some other full directory paths I have.
I tried calling toURL() on the DirectoryEntry and it returned an empty string.
A bit about my project: I want to write an iTunes library synchronizer as a Chrome Packaged App. The iTunes library XML file contains full paths like file://localhost/D:/David/Music/Bob/Bob%20Album/01%20Bob.mp3. The user will give my app access to his music folders, and I want to be able to tell if he gave me access to the right folders.
The only full paths available are those returned by getDisplayPath.
The mediaGalleries API may be a better fit for your project: http://developer.chrome.com/apps/mediaGalleries.html#iTunes.
If you have a full path path/to/the/file.mp3 and you want to load the file directly with tis you can do it but the solution is not perfect. Use the chrome.mediaGalleries API to ask where the user saves his music (in your case), then you can write a loop who check if one of the path repository is equal to a gallery.
For example if you got a file path/to/the/file.mp3 from the xml file and your galleries list looks like ["D", "to", "Videos"], write a function to check if each component of the file's path is a gallery. In this case your code will find "to", so you can launch a second function who use "the/file.mp3".
The second function has to use the given path and find if the gallery contains the right folders and finally the right file (use this example by Google). In the case you're trying to find "the/file.mp3" with the gallery to your loop has to find a directory named "the" then "file.mp3" (write a recursive function), if you find the file open it, otherwise come back to the first function if you haven't check all the galleries or all the path's component.
This is currently (2014-01-10) not a feature of Chrome, but I have suggested it and they are working on it:
https://code.google.com/p/chromium/issues/detail?id=322952
I want to control parameters of .exe file when it runs again. For example
DocumentViewer.exe -DocID=1097911 -DBName "TEST.WORLD" -DBUname "DB1" -DBPwd "DB1" -AccessUserName "user1" -AccessPassword "pass1" -AccessPath "C:\..."
While this .exe file is running if user runs a new instance of .exe file with different DocID, I want to inform user "You should complete your work with DocID:1097911 to open new document".
I don't understand where should I get second instance's parameters to control with the first one?
You'll need some type of data store in order to save the parameters. Then after the instance is terminated you need to clear out the data store. In this example i'm going to use a text file for simplicity.
when an instance of your .exe is started, first read from a centralized text file. If the text file is empty then update the file with the instance's parameters and continue starting. If its not empty then notify the user with the DocId from the text file.
When I download my program from my website to my windows 2003 machine, it has a block on it and you have to right click on the exe, then properties, then select the button "Unblock".
I would like to add detection in my installer for when the file is blocked and hence doesn't have enough permissions.
But I can't eaisly reproduce getting my exe in this state where it needs to be unblocked.
How can I get the unblock to appear on my exe so I can test this functionality?
This is done using NTFS File Streams. There is a stream named "Zone.Identifier" added to downloaded files. When IE7 downloads certain types of file that stream contains:
[ZoneTransfer]
ZoneId=3
The simplest way to set it is to create a text file with those contents in it, and use more to add it to the alternate stream.
Zone.Identifier.txt:
[ZoneTransfer]
ZoneId=3
Command:
more Zone.Identifier.txt > file.exe:Zone.Identifier
Then, the way for you to check it would be to try to open the Zone.Identifier stream and look for ZoneId=3, or simply assume that if the stream exists at all that your user will receive that warning.
It's also important to note that this has nothing to do with permissions. Administrators see the same warning; it's to do entirely with the source and type of file. The entire stream goes away when users uncheck the "Always ask before opening this file" box and then click Run.
There is a supported API for this, documented on MSDN. Search on MSDN for "Persistent Zone Identifier Object". Basically you CoCreateInstance with CLSID_PersistentZoneIdentifier and request an IPersistFile interface. You then call IPersistFile::Load with the name of the file in question. Next, QI for an IZoneIdentifier interface and use IZoneIdentifier::GetId to obtain the zone of the file. If there was no "mark of the web", you should get URLZONE_LOCAL_MACHINE. The ZoneId of 3 mentioned in the other reply is URLZONE_INTERNET. (The enumeration is called URLZONE and is also documented on MSDN, or see sdk\inc\urlmon.h.) You can remove or change the "mark of the web" by calling IZoneIdentifier::Remove or IZoneIdentifier::SetId and then call IPersistFile::Save. There are more details about all of this on MSDN. Good luck!
Thanks for this it helped me a lot.
You can make the process even easier if you create a batch file with the contents.
echo [ZoneTransfer] > Zone.Identifier
echo ZoneId=3 >> Zone.Identifier
more Zone.Identifier > %1:Zone.Identifier
This will generate the Zone.Identifier for you and mark the file accordingly.
To run it just supply the file name e.g. if the file is called mark.bat
mark.bat myfile.txt