I am trying to parse a giant log file using node.js, the file does not seem to get '\n' but when I do set list in vi it shows me '$" at the end of every line, does anyone know what that is. I means can I split a string on that.
I would recommend checking out your file via
cat -v -e
which will show you all unprintable characters and line endings.
It happens when you do set list, so you should read :h 'list' instead of asking this here. Everything what you need to know about this $ is stated in the help.
Second question (splitting string on end-of-line) is answered in :h getline(). I also doubt that file really does not have a NL so write here how did you came to conclusion «the file does not seem to get '\n'».
Related
I've got a file that contains lines that look like this in vim:
^[[0;32msalt-2016.3.2-1.el6.noarch^[[0;0m^M
which look like this in more:
salt-2016.3.2-1.el6.noarch
I would like to produce a copy of this file that only contains the displayed characters as more shows them. I tried piping it through dos2unix but it refuses to do anything, complaining that "dos2unix: Binary symbol 0x1B found at line 2".
Probably I could achieve what I want with some sed statements, but I'm wondering whether there is a linux/unix utility that will take output from more or cat and produce a file that contains only the whitespace and text as displayed?
There's something called ansifilter which does exactly this. I tested it out on my file and it works.
I got some Postgres table dumps from somebody using pgAdmin3 on Windows. (Blech.) First of all, it has a whole bunch of extra crap at the top of the file that I've had to get rid of-- things like "toc.dat" without comments, etc.
I've resorted to editing them by hand to get them in workable format to be imported, because as it stands they are somewhat garbled; for the most part I've succeeded, but when I open them in emacs, for example, they tend to be littered with the following character:
^#
and sometimes just alot of:
###
I haven't figured out how to remove them using sed or awk, mainly because I have no idea what they are (I don't think they are null characters) or even how to search for them in emacs. They show up as red for 'unprintable' characters. (Screenshot above.) They also don't seem to be printed to the terminal when I cat the file or when I open it in my OS X Text editor, but they certainly cause errors when I try to import the file in to postgres using
psql mydatabase < table.backup
unless I edit them all out.
Anybody have any idea of a good way to get rid of these short of editing them by hand? I've tried in place sed and also tried using tr, but to no effect-- perhaps I'm looking for the wrong thing. (As I'm sure you are aware, trying to google for '^#' is futile!)
Just was wondering if anybody had come across this at all because it's going to eat at me unless I figure it out...
Thanks!
Those are null characters. You can remove them with:
tr -d '\000' < file1 > file2
where the -d parameter is telling tr to remove characters with the octal value 000.
I found the tr command on this forum post, so some credit goes to them.
I might suggest acquiring access to a Windows machine (never thought I'd say that), loading the original dumps they gave you, and exporting in some other formats to see if you can avoid the problem altogether. Which to me seems safer than running any for of sed or tr on a database dump before importing. Good luck!
I have a requirement which I am sure I can do in vi (plenty of other solutions I am sure), and this is it.
I have a file that looks like this
1234 Some Text HERE rest of line
1235 Some Other Text HERE rest of line
What I want to do is delete text from, and including the word HERE to the end of the line, leaving me with this;
1234 Some Text
1235 Some Other Text
I have done search and replace things in vi, but am not sure how to do a search then run a command.
Any help, as always is greatly appreciated.
Thanks in anticipation
How about that:
:%s/HERE.*//
This pattern replaces the part of the line starting with HERE and replaces it with nothing.
The combination of d$ will delete from the cursor to the end of the current line. So if you're searching using the / command, it'll be...:
/HERE<enter>d$
As always, in vi there are many ways to skin this particular cat. As has already been stated, a simple solution to exactly this problem is:
:%s/HERE.*//
However, to answer your more general question:
I have done search and replace things in vi, but am not sure how to do a search then run a command.
you want the :g[lobal] command:
:[range]g[lobal]/{pattern}/[cmd]
Execute the Ex command [cmd] (default ":p") on the
lines within [range] where {pattern} matches.
This would actually be more long-winded for your exact example, but for tasks such as deleting lines that match a specific pattern, it is considerably more concise.
e.g. if you wanted to instead delete all lines that contain HERE, you would run:
:g/HERE/d
Factoid: this command form is the origin of grep's name: g/re/p, shorthand for global/{regex}/print.
This can be done using sed also:
sed "s/HERE.*//"
Example:
echo "this is a test HERE test" | sed "s/HERE.*//"
Result:
this is a test
There is a lot of documentation about vim's quickfix but I would like to know:
How I can I use a text file (or copy+paste from a terminal), and put this into a quick-fix list in vim?
All the docs I found so far assume that you run make or vimgrep, but in this case I just want to use a text file created elsewhere.
You're looking for :cfile / :cgetfile:
:cf[ile][!] [errorfile] Read the error file and jump to the first error.
The file you're reading naturally must be in a format that can be parsed by the 'errorformat' option.
One way to do it is to change makeprog to cat from your text file, pretty sure there's a better way to do it though.
The following symbol shows up when i view my file in vim.
---<snip>----
^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#
^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#^#fstalone
---<snip>-----
The file that I create is by redirecting stdout and stderr of my utility, like this: #./my_util > util.log 2>&1. This file tend to grow quite huge ( ~4 MB )
What is this symbol?
How to get rid of it?
That is the null character, in a format (which Vim uses a lot, as you've probably noticed) called caret notation. Basically, somehow you're getting bytes full of zeros into your file.
Since we don't know what your utility is doing, if it's the culprit, you'll need to show us some code if you want us to help. Otherwise, if you just want to remove the characters from your file, use a substitution:
%s/<Ctrl-V><Ctrl-J>//g
Ctrl-V marks the beginning of an escape sequence. After pressing Ctrl-J as well, you should see ^# appear in your command. Thus, as you guessed, Ctrl-V, Ctrl-J is one escape sequence for the null character.
None of the above worked for me. I had a file with '^#' at the end of some lines that I wanted to replace. I managed the substitute it by searching for '[\x0]' using:
%s/[\x0]//g
I hope it saves someone an hour of their life.
There's an explanation here that I will go back to read when I'm not so busy:
A discussion with a better explanation
^# shows up when you try to open a non text file in vim. For example if you open a exe file or an image file ^# is shown which is a non-readable character. Try opening the file in some other editor and see the result