NodeJS RTF ANSI Find and Replace Words With Special Chars - node.js

Question

I have a find and replace script that works no problem when the words don't have any special characters. However, there will be a lot of times where there will be special characters since it's finding names. As of now this is breaking the script.
The script looks for {<some-text>} and attempts to replace the contents (as well as remove the braces).
Example:
text.rtf
Here's a name with special char {Kotouč}
script.ts
import * as fs from "fs";
// Ingest the rtf file.
const content: string = fs.readFileSync("./text.rtf", "utf8");
console.log("content::\n", content);
// The string we are looking to match in file text.
const plainText: string = "{Kotouč}";
// Look for all text that matches the patter `{TEXT_HERE}`.
const anyMatchPattern: RegExp = /{(.*?)}/gi;
const matches: string[] = content.match(anyMatchPattern) || [];
const matchesLen: number = matches.length;
for (let i: number = 0; i < matchesLen; i++) {
// It correctly identifies the targeted text.
const currMatch: string = matches[i];
const isRtfMetadata: boolean = currMatch.endsWith(";}");
if (isRtfMetadata) {
continue;
}
// Here I need a way to escape `plainText` string so that it matches the source.
console.log("currMatch::", currMatch);
console.log("currMatch === plainText::", currMatch === plainText);
if (currMatch === plainText) {
const newContent: string = content.replace(currMatch, "IT_WORKS!");
console.log("newContent:", newContent);
}
}
output
content::
{\rtf1\ansi\ansicpg1252\cocoartf1671\cocoasubrtf600
{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
{\colortbl;\red255\green255\blue255;}
{\*\expandedcolortbl;;}
\margl1440\margr1440\vieww10800\viewh8400\viewkind0
\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural\partightenfactor0
\f0\fs24 \cf0 Here's a name with special char \{Kotou\uc0\u269 \}.}
currMatch:: {Kotou\uc0\u269 \}
currMatch === plainText:: false
It looks like ANSI escaping, and I've tried using jsesc but that produces a different string, {Kotou\u010D} instead of what the document produces {Kotou\uc0\u269 \}.
How can I dynamically escape the plainText string variable so that it matches what is found in the document?

Answer 1

What I needed was to deepen my knowledge on rtf formatting as well as general text encoding.
The raw RTF text read from the file gives us a few hints:
{\rtf1\ansi\ansicpg1252\cocoartf1671\cocoasubrtf600...
This part of the rtf file metadata tells us a few things.
It is using RTF file formatting version 1. The encoding is ANSI, and specifically cpg1252, also known as Windows-1252 or CP-1252 which is:
...a single-byte character encoding of the Latin alphabet
(source)
The valuable piece of information from that is that we know it is using the Latin alphabet, this will be used later.
Knowing the specific RTF version used I stumbled upon the RTF 1.5 Spec
A quick search on that spec for one of the escape sequences that I was looking into revealed that it was an RTF specific escape control sequence, that is \uc0. So knowing that I was able to then parse what I was really after, \u269. Now I knew it was unicode and had a good hunch that the \u269 stood for unicode character code 269. So I look that up...
The \u269 (char code 269) shows up on this page to confirm. Now I know the character set and what needs done to get the equivalent plain text (unescaped), and there's a basic SO post I used here to get the function started.
Using all this knowledge I was able to piece it together from there. Here's the full corrected script and it's output:
script.ts
import * as fs from "fs";
// Match RTF unicode control sequence: http://www.biblioscape.com/rtf15_spec.htm
const unicodeControlReg: RegExp = /\\uc0\\u/g;
// Extracts the unicode character from an escape sequence with handling for rtf.
const matchEscapedChars: RegExp = /\\uc0\\u(\d{2,6})|\\u(\d{2,6})/g;
/**
* Util function to strip junk characters from string for comparison.
* #param {string} str
* #returns {string}
*/
const cleanupRtfStr = (str: string): string => {
return str
.replace(/\s/g, "")
.replace(/\\/g, "");
};
/**
* Detects escaped unicode and looks up the character by that code.
* #param {string} str
* #returns {string}
*/
const unescapeString = (str: string): string => {
const unescaped = str.replace(matchEscapedChars, (cc: string) => {
const stripped: string = cc.replace(unicodeControlReg, "");
const charCode: number = Number(stripped);
// See unicode character codes here:
// https://unicodelookup.com/#latin/11
return String.fromCharCode(charCode);
});
// Remove all whitespace.
return unescaped;
};
// Ingest the rtf file.
const content: string = fs.readFileSync("./src/TEST.rtf", "binary");
console.log("content::\n", content);
// The string we are looking to match in file text.
const plainText: string = "{Kotouč}";
// Look for all text that matches the pattern `{TEXT_HERE}`.
const anyMatchPattern: RegExp = /{(.*?)}/gi;
const matches: string[] = content.match(anyMatchPattern) || [];
const matchesLen: number = matches.length;
for (let i: number = 0; i < matchesLen; i++) {
const currMatch: string = matches[i];
const isRtfMetadata: boolean = currMatch.endsWith(";}");
if (isRtfMetadata) {
continue;
}
if (currMatch === plainText) {
const newContent: string = content.replace(currMatch, "IT_WORKS!");
console.log("\n\nnewContent:", newContent);
break;
}
const unescapedMatch: string = unescapeString(currMatch);
const cleanedMatch: string = cleanupRtfStr(unescapedMatch);
if (cleanedMatch === plainText) {
const newContent: string = content.replace(currMatch, "IT_WORKS_UNESCAPED!");
console.log("\n\nnewContent:", newContent);
break;
}
}
output
content::
{\rtf1\ansi\ansicpg1252\cocoartf1671\cocoasubrtf600
{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
{\colortbl;\red255\green255\blue255;}
{\*\expandedcolortbl;;}
\margl1440\margr1440\vieww10800\viewh8400\viewkind0
\pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\pardirnatural\partightenfactor0
\f0\fs24 \cf0 Here\'92s a name with special char \{Kotou\uc0\u269 \}}
newContent: {\rtf1\ansi\ansicpg1252\cocoartf1671\cocoasubrtf600
{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
{\colortbl;\red255\green255\blue255;}
{\*\expandedcolortbl;;}
\margl1440\margr1440\vieww10800\viewh8400\viewkind0
\pard\tx560\tx1120\tx1680\tx2240\tx2800\tx3360\tx3920\tx4480\tx5040\tx5600\tx6160\tx6720\pardirnatural\partightenfactor0
\f0\fs24 \cf0 Here\'92s a name with special char \IT_WORKS_UNESCAPED!}
Hopefully that helps others that aren't familiar with character encoding/escaping and it's uses in rtf formatted documents!