I need to insert a generated file into an exe at the time of download. Currently, I create an "empty" file (filled with a repeating character) and package that with the exe. When it comes time to download, I look at the bytes for the installer, find the file by looking for the repeating character, and insert the generated file.
This process however is not working. The repeating character just does not show in the bytes. But I am certain the file is there as it is unpacked if I run the exe. Am I doing something wrong or is inserting a file into an exe even possible?
Also note that I am using Inno Setup Script v5.5.1 to compile the project into an exe.
If you want to change the contents of a file specified in a [Files] entry and compiled into the setup executable, then you must:
Make a dummy file that is at least as large as the largest content you will want to insert.
Fill the file (or at least the first 64 bytes or so) with something unique and easily distinguishable.
Mark its [Files] entry with the "nocompression noencryption dontverifychecksum" flags.
You should then be able to scan the resulting executable for the marker in #2 and then substitute the data that you want. Note however that doing this might invalidate any digital signature on the setup file, although I haven't tested this to be sure.
Note that if the content you are inserting is smaller than the dummy file size, the extra bytes will still remain on the end of your inserted content. So whatever reads the file will have to have some way to ignore that or to recognise the end of the interesting content.
So, if your are making changes in the existing exe file, and if the text is not much, you can probably use some hex editor and make changes at desired location. If text is more , you might want to include some meaningless bytes, just as fillers.
Related
I have few lines in a file that's been loaded into list. In the file is line that starts with EKO-1223... I would like to get this line, so I am using a while loop and iterate over the lines from the list. I am using nPos = StrFind(svLine, "EKO") but the nPos is < 0 so it does not find the result, why?
If you want a good answer, you'll need to provide more details on how you get the text from the file into svLine, as well as anything you know about the file's encoding, etc. (If you know nothing about encodings, a hex dump of the first few bytes of the file, as well as those including the EKO- may suffice to identify it.)
My guess is either you haven't properly loaded svLine at all, or that the encoding was misidentified, and thus svLine contains something like "E\0K\0O\0..." or "䭅ⵏ" (that is "\x4b45\x2d4f" in C notation). Can you confirm with a message box or in the debugger?
One alternative you could consider is calling FileGrep. This could help if your code didn't load the file at all, but is unlikely to handle encodings any better. If it's improperly detected encoding and you can change the file, ensure the file has the correct BOM for its encoding. But if you don't control the file, I'm not sure what to recommend. Binary reads and manual decoding (possibly leveraging Kernel32.MultiByteToWideChar) might be your best bet.
I have a project that involves extracting data from a database into a text file, and then ingesting it into Hadoop. So i want to create a shell script that NiFi can run to automatically to check if a text file is extracted and ingest it, but I need to make sure that the whole data has been extracted first before ingesting it. Meaning I would need to check that the text file has an EOF, how do I do that?
Don't have any code as of yet, I have very little knowledge writing shell scripts.
While creating the file, use a different name. Rename it to the expected name once the extraction is done. Then, the other process can start its work once the file exists.
EOF is not something that actually gets put in the text file - in fact, there isn't really any EOF value. EOF or end-of-file is a condition that occurs when you try to consume input from a source that has none to give.
There is no general marker you can look for in your text files that will tell you whether they are complete. You'll need to make your script indicate when a given chunk of data has been extracted in some other way. There are many possibilities; you could change the name of the file as choroba suggested, or you could create a lock file and remove it once the data extraction is done, or you could have your extraction program write a distinctive sequence of bytes to the file at the end, or so on.
I have a 23GB file and I would like to edit the 23rd line, but I have only 200 MB RAM available on the server. I do not want to open the file entirely because I have left only 20GB available disk space.
How can I do this. I tried to use head, tail sed but it seems it creates a temporary file. Is it possible to do it without a temporary file?
The solution is to edit the file with a hex editor. Hex editors are built to handle huge files, even whole disks and partitions.
You may find hexedit (ncurses based) or ghex (Gnome/Gtk based) useful. They are common utilities, therefore you will most probably find them in your distributions's repo.
All hex editors I have used, use a twin panel view with the left panel showing the bytes of the file in Hex, and the right panel trying to show an Ascii representation when that is possible.
In order to find and edit your 23rd line:
sed -n '23p' my_huge_dump.sql : Will print the contents of this line
sed -n '23p' my_huge_dump.sql | od -A n -t x1 : Will print the contents of this line in hexadecimal format.
or open the file with less -N my_huge_dump.sql and view the contents of line 23. (-N in less enables line numbering)
Now, knowing the content of the 23rd line:
If the text of this line is somewhat unique and different from surrounding lines, you may find it from the right (ascii) panel and navigate to this line with the arrows. In hexedit you use the Tab key to move between the Hex and Ascii panels. In gHex you can use your mouse as well. You may also search for the string you're interested: Move to the Ascii panel and press / in hexedit or use the menu in gHex.
If the line you want to edit has similar contents to other lines and you can't find it in the ascii panel, then you must count the "newline" separators to find the 23rd line. New lines (LF) are represented as 0A in hex. In the ASCII panel, new lines are represented as dots .
Then assuming you found the line you want to edit, you have the following options:
Hopefully, the new content of the 23rd line is shorter or equal in length to the existing content (so you won't need to grow and move the whole file). In this case, you have to enter the Fill-mode i.e. the mode in which you overwrite existing content typing over the old text. This is the default mode in both gHex and hexedit. Move to the location you want to edit and start typing. Pressing Backspace will undo your changes. If the new content is shorter than the existing, you may fill up the line with spaces to avoid truncating the file.
If the new content is longer than the existing one in this line, then you have to enter the Insert mode. You can do that using the Menu in gHex. In hexedit you have to use the EscI keybinding. Then start typing and the new characters will be appended in the current location.
In the first case, it is guaranteed that the editing and saving of the file will be instantaneous since an in-place edit will happen. In the later case, I'm not sure how the growing in size and the moving of following bytes will be handled, but I hope the filesystem uses a larger non-continuous block to move some of the contents and not move the whole file.
If you're happy with your changes, save the file:
Use the menu in gHex
Use Ctrlx in hexedit and answer (Y)es when questioned about whether to save the changes.
Always make sure you have a backup in place!
EDIT: I found out that gHex isn't suitable for your situation, since it tries to load the whole file in memory. hexedit will serve you fine. However, if you want a graphical editor like gHex, but with partial file loading capabilities, you may try wxHexEditor. Check also the Comparison of Hex editors page in Wikipedia.
Liquid Studio Community Edition contains a Large File Editor which can open and edit Terra-byte files on low spec machines, and its free.
It requires enough disk space to copy the file (when writing it back out), but hardly requires any memory.
I have a file with a list of properties.
Name
Description
BogusProperty_the_first
The full file has some 200 properties
I also have an xml file that references properties in the previous list, containing entries like;
<Item value="#Name#" length="32" description="Name" />
I want to remove from the first file all entries that are/are not present in the second file.
I do not need a perfect fit, it's OK if I treat some entries as being present in the second file when in fact they are not, so it's sufficient to test that "Description" occurs somewhere in the second file, I don't need to test that value="#Description#" occurs in a tag at the appropriate place in the DOM.
It would be bad to treat entries in the first file as not being in the second file if in fact they where.
The solution does not need to be completely automated or a single button click, but I do not want to check each item in the first file separately.
I am using notepad++, but would be open to using other tools if applicable.
The problem is small enough that writing a separate program to handle it, while straight forward, would not be worth it.
While writing the question I realised that notepad++ can solve this by copy pasting the second file into a copy of the first file.
The procedure I used was the following;
Write a seperate line in the first file with text that does not occur in the either of the two files. In my case I used asdf1234.
Copy the contents of the second file into the first.
Search for the following regexp with ".matches newlines" checked.
(?:\n|\r)([^\n\r]+)(?=(?:\n|\r).*asdf1234.*\1)
Replace with nothing.
???
Profit
To keep the entries that do occur in the second file use (?:\n|\r)([^\n\r]+)(?=(?:\n|\r))(?!.*asdf1234.*\1) to search.
I want to use a algorithm to reduce memory used to save the particular text file.I don't really know how text is stored but i have an idea in mind.
Would it be better to extend a open source text editor (if yes than which one) or write a text editor myself.
It would be nice if someone could also give me a link or tutorial to some basics on how text editors work and the way data is stored.
Edited to add
To clarify, what I wanted to do is instead of saving duplicates of a word make a hash table and store the address where it needs to be placed.
That way I wouldn't be storing the duplicates.
This would have become specific to a particular text editor.
Update
thanks everyone I got what all of you'll are trying to say. Anyways all i wanted to do is instead of saving duplicates of a word make a hash table and store the address where it needs to be placed.
This was i wouldn't be storing the duplicates.
Yes and this would have become specific to a particular text editor. never realized that.
I want to use a algorithm to reduce memory used to save the particular text file
If you did this you would no longer have a text editor, but instead you would have created some sort of binary file editor.
The whole point of the text file format is that it is universal, meaning any text file can be open in any other text editor.
Emacs handles compression transparently. Just create a text file with .gz extension. Emacs will automatically compress contents of the file during save operation, and decompress when you open the file next time.
Text is basically stored as-is. i.e., every character takes up a byte or two (wide chars), and there is no conversion done on it when it's saved. It might add an end-of-file character or something though. Don't try coming up with your own algorithm to compress these files. That's why zip-files and other archives were created. They're really good at compressing text. If you wanted to add these feature to your text-editor, you'd have to add some sort of post-save hook to zip it, and then put a hook on the open command to unzip it. Unless you wanted to do it by hand every time. Don't try writing the text editor yourself from scratch, unless (maybe) you're writing notepad. Text editors with syntax highlighting aren't very easy to make, even with the proper libraries. I'd say write a plugin for something like Visual Studio or what have you. Or find an open-source text editor.