Let me know how to delete the null line using sqlldr, ctl.
And I wanna know how to remove the last two line of files.
There are null lines in tail, that is the last 1~2 line.
Plus I cannot know the last line number.
wait to reply😥
You need to either pre-process the file and remove blank lines before running sqlldr via a wrapper script, or more common, just load all rows from the file into a staging table, then call a PL/SQL script from there to load into the main table.
Pre-processing alters the main file so that is usually not a good idea unless of course you make an archive copy first.
Using a staging table is more common as that way all rows from the file are available and you can select the rows you want, transforming, validating, etc the data on the way into the main table.
Related
Long story short, after a crash course in Python/BeautifulSoup, I managed to create a script to take an input text file that contains a list of URLs (1 on each line), scrape the URL, and write output to a database. There are some cases where I want an error to exit the script (including some trapped errors as well as unexpected), but as the list of URLs to scrape is pretty large, it would be handy if I could edit the input text file (or create a copy and edit that) to remove each URL as it is successfully processed. The idea being that if the script exits (by trap or crash), I'd have a list of the URLs left to be processed. Is something like this possible? I can find code samples to edit the text file, but I'm getting stuck at how to take out the row just processed.
Finally came across the post here that achieves the answer, though I'm not positive it's the most efficient way as it's reading the entire file and saving each time, but that may be the best that can be done in Python. In my case, the file is in the 1200 lines range, so it easily fits into memory.
I am writing a REXX program which will update a PS dataset. I can edit a particular line using my REXX code. But I would want a code to insert a particular string after a particular line.
For Example: My PS dataset has 100 lines. I want to insert a text "ABCDE" after 44th line (in 45th line) which will increase the total lines of the file to 101 lines. The remaining lines should remain unchanged. Is this possible using REXX?
Independent of REXX you need to effectively read the old dataset and write it out to a new file and add your new record (string) to the output file and then write the rest. There is no way to “insert” a record in a Physical Sequential (PS) dataset. At the end you would delete the old and rename the newly created file to the old name.
Another option would be to use a generation dataset group (GDG) and read the current (0) and create the new (+1) as the output. This way you still are referring to the same dataset name for others to reference.
What #Hogstrom suggests is a good solution to the problem you describe. In the interest of completeness, here is a solution that may be necessary under extreme circumstances.
Create an edit macro...
/*REXX*/
ADDRESS ISREDIT 'MACRO NOPROCESS'
aLine = 'ABCDE'
ADDRESS ISREDIT 'LINE_AFTER 44 = DATALINE (ALINE)'
...and run ISPF edit in batch, executing this macro.
The JCL to run ISPF in batch is shop-specific, but many shops have created a cataloged procedure to do so.
If you are willing to copy your dataset to the z/Unix file system, you could also use sed or awk to make your changes.
I'm not recommending any of this, I'm just pointing out it can be done if #Hogstrom's solution won't work for you for some reason.
I have a project that involves extracting data from a database into a text file, and then ingesting it into Hadoop. So i want to create a shell script that NiFi can run to automatically to check if a text file is extracted and ingest it, but I need to make sure that the whole data has been extracted first before ingesting it. Meaning I would need to check that the text file has an EOF, how do I do that?
Don't have any code as of yet, I have very little knowledge writing shell scripts.
While creating the file, use a different name. Rename it to the expected name once the extraction is done. Then, the other process can start its work once the file exists.
EOF is not something that actually gets put in the text file - in fact, there isn't really any EOF value. EOF or end-of-file is a condition that occurs when you try to consume input from a source that has none to give.
There is no general marker you can look for in your text files that will tell you whether they are complete. You'll need to make your script indicate when a given chunk of data has been extracted in some other way. There are many possibilities; you could change the name of the file as choroba suggested, or you could create a lock file and remove it once the data extraction is done, or you could have your extraction program write a distinctive sequence of bytes to the file at the end, or so on.
I have a file with a list of properties.
Name
Description
BogusProperty_the_first
The full file has some 200 properties
I also have an xml file that references properties in the previous list, containing entries like;
<Item value="#Name#" length="32" description="Name" />
I want to remove from the first file all entries that are/are not present in the second file.
I do not need a perfect fit, it's OK if I treat some entries as being present in the second file when in fact they are not, so it's sufficient to test that "Description" occurs somewhere in the second file, I don't need to test that value="#Description#" occurs in a tag at the appropriate place in the DOM.
It would be bad to treat entries in the first file as not being in the second file if in fact they where.
The solution does not need to be completely automated or a single button click, but I do not want to check each item in the first file separately.
I am using notepad++, but would be open to using other tools if applicable.
The problem is small enough that writing a separate program to handle it, while straight forward, would not be worth it.
While writing the question I realised that notepad++ can solve this by copy pasting the second file into a copy of the first file.
The procedure I used was the following;
Write a seperate line in the first file with text that does not occur in the either of the two files. In my case I used asdf1234.
Copy the contents of the second file into the first.
Search for the following regexp with ".matches newlines" checked.
(?:\n|\r)([^\n\r]+)(?=(?:\n|\r).*asdf1234.*\1)
Replace with nothing.
???
Profit
To keep the entries that do occur in the second file use (?:\n|\r)([^\n\r]+)(?=(?:\n|\r))(?!.*asdf1234.*\1) to search.
I need to insert a generated file into an exe at the time of download. Currently, I create an "empty" file (filled with a repeating character) and package that with the exe. When it comes time to download, I look at the bytes for the installer, find the file by looking for the repeating character, and insert the generated file.
This process however is not working. The repeating character just does not show in the bytes. But I am certain the file is there as it is unpacked if I run the exe. Am I doing something wrong or is inserting a file into an exe even possible?
Also note that I am using Inno Setup Script v5.5.1 to compile the project into an exe.
If you want to change the contents of a file specified in a [Files] entry and compiled into the setup executable, then you must:
Make a dummy file that is at least as large as the largest content you will want to insert.
Fill the file (or at least the first 64 bytes or so) with something unique and easily distinguishable.
Mark its [Files] entry with the "nocompression noencryption dontverifychecksum" flags.
You should then be able to scan the resulting executable for the marker in #2 and then substitute the data that you want. Note however that doing this might invalidate any digital signature on the setup file, although I haven't tested this to be sure.
Note that if the content you are inserting is smaller than the dummy file size, the extra bytes will still remain on the end of your inserted content. So whatever reads the file will have to have some way to ignore that or to recognise the end of the interesting content.
So, if your are making changes in the existing exe file, and if the text is not much, you can probably use some hex editor and make changes at desired location. If text is more , you might want to include some meaningless bytes, just as fillers.