RFC 4566 is the controlling RFC for SDP syntax.
It states in Section 5 - SDP Specification that:
An SDP session description consists of a number of lines of text of
the form:
<type>=<value>
where <type> MUST be exactly one case-significant character and
<value> is structured text whose format depends on <type>. In
general, <value> is either a number of fields delimited by a single
space character or a free format string, and is case-significant
unless a specific field defines otherwise. Whitespace MUST NOT be
used on either side of the "=" sign.
However, nowhere is it clear whether there can be whitespace before the case-significant character.
Section 9.0 which provides the BNF grammar is also ambiguous on this issue. All SDP entriess I have seen appear to start the attribute lines from the first position but is whitespace allowed at the start of an SDP entry is the question.
The answer provided to a somewhat similar but definitely different question I had asked earlier sheds some light but is not definitive on this particular issue.
Spaces before the case-significant character is not allowed. The BNF/ABNF does not show that you can add spaces before the lines defined in session-description. They even explicit say which letter you have to use like v=....
Related
Yesterday I added an answer to How to add a line break to text in UI5?. While trying to specify the question with more tags, I realized that there are two similar tags available on Stack Overflow: newline and line-breaks.
The differentiation between LF and CR is pretty clear. But what is the difference between the terms "newline" and "line break"? Aren't they synonym to each other?
The newline excerpt here on Stack Overflow says:
Newline refers to [...] a line break.
... while the disambiguation page of "Line break" from Wikipedia says:
Line break may refer to [...] newline.
And from that "Newline" page:
To denote a single line break, Unix programs use line feed [...] while most programs common to [...] Windows use carriage return+line feed.
Are there any technical standards or guidelines that make a clear distinction between those two terms in the software industry? Or can they be used interchangeably, having no technical difference?
My current assumption is that there is no difference: "line break" describes the result from either soft return (⇧ Shift+↵ Enter) or hard return (↵ Enter), whereas "newline" is a technical term for "line break" but has the same result.
I interpret line break as a semantic meaning. What you tell a typographer when you see a \n. C language "new line" means a line break. This is like A (0x41) in ASCII is the upper case of Latin letter a. So a name, a code, and a meaning.
It is like "SP" (\u0020) is a space character. There are many other space characters: C recognizes also TAB as space character. HTML recognize new line as space character (and not as a line break (but in <pre>).
CR and LF were defined by ASCII, and used together because typewritter usually worked in such manner, and to give some more time to move (they were connected serially, so with a timed inputs). You may find the scanned document of old ASCII about the meaning of control characters, but that changed a lot. \0 is now a end of string, before it was just a filling character (or to quit a sequence). Apple used CR, Unix LF as what typographers used as line break.
If you want a list of names and aliases, Unicode provides such names (but as usually, there were errors, so BELL is now used for a emoji, and so the old alias of BEL is not more valid, for control code). There are also several sources about standardization of control codes (mainly used for escape sequances, and so). But also this is not fully follower. Terminals tend to have quirks.
I am trying to search for multiple strings in a text log alltogether with the following pattern:
s(n)KEY: some data
s(n)Measurement: some data
s(n)Units: some data
Where s(n) is the number of spaces that varies. KEY will change at every iteration in the loop as it comes from the .ini file. As an example see the following snippet the of log:
WHITE On Axis Lum_010 OPTICAL_TEST_01 some.seq
WHITE On Axis Lum_010 Failed
Bezel1 Luminance-Light Source: Passed
Measurement: 148.41
Units: fc
WHITE On Axis Lum_010: Failed
Measurement: 197.5
Units: fL
In this case, I only want to detect when the key (WHITE On Axis Lum_010) appears along with Measurement and I don't want to detect if it appears anywhere else in the log. My ultimate goal is to get the measurement and unit data from file.
Any help will be greatly appreciated. Thank you, Rav.
I'd do it similar to Salome, using regular expressions. Since those are a little tricky, I have a test VI for them:
The RegEx is:
^\s{2}(.*?):\s*(\S*)\n\s*Measurement:\s*(\S*)\n\s*Units:\s*(\S*)
and means:
^ Find a beginning of a line
\s{2} followed by exactly two whitespaces
(.*?) followed by multible characters
: followed by a ':'
\s* followed by several whitespaces
(\S*) followed by several non-whitespaces
\n followed by a newLine
\s* followed by several whitespaces
Measurement: followed by this string
\s* followed by several whitespaces
(\S*) followed by several non-whitespaces
\n followed by a newLine
... and the same for the 'Unit'
The parentheses denote groups, and allow to easily collect the interesting parts of the string.
The RegEx string might need more tuning if the format of the data is not as expected, but this is a starting point.
To find more data in your string, put this in a while loop and use a shift register to feed the offset past match into the offset of the next iteration, and stop if it's =-1.
It's easier to search through and to implement.
LabVIEW also has VIs to create and manage JSONs.
Alternatively, you could use Regular Expressions in a while-loop to look if it exists in your log, maybe something like this:
WHITE On Axis Lum_010:(\s)*((Failed)|(Pass))\n(\s)+Measurement:(\s)*[0-9]*((\.)[0-9]*){0,1}\n(\s)*Units:\s*\w*
Then you can split the string or pick lines and take the information.
But I would not recommend that, as it is impractical to change and not useful if you want to use the code for other keys.
I hope it helps you :)
I have a .html file that is working perfectly fine but for some reason Sublime 3 decides that it has invalid code, check the image below:
Any idea why that's happening and how to fix it without having to modify the code?
The HTML5 spec states (my emphasis):
Comments must start with the four character sequence U+003C LESS-THAN SIGN, U+0021 EXCLAMATION MARK, U+002D HYPHEN-MINUS, U+002D HYPHEN-MINUS (<!--). Following this sequence, the comment may have text, with the additional restriction that the text must not start with a single > (U+003E) character, nor start with a U+002D HYPHEN-MINUS character (-) followed by a > (U+003E) character,
nor contain two consecutive U+002D HYPHEN-MINUS characters (--),
nor end with a U+002D HYPHEN-MINUS character (-). Finally, the comment must be ended by the three character sequence U+002D HYPHEN-MINUS, U+002D HYPHEN-MINUS, U+003E GREATER-THAN SIGN (-->).
So that's why it's complaining. As to how to fix it without changing the code, that's trickier.
Your contention that it works is no different really to C developers wondering why they need to worry about undefined behaviour because the code they wrote works fine. The fact that it works fine in one particular implementation is not relevant to portable code.
My advice is to actually change the code. It's not valid, after all, and any browser (current or future) would be well within its rights to simply reject it.
As an aside after some historical digging, it appears this is not allowed because SGML, on which HTML was based, had slightly different rules regarding comment.
On sensing the <!-- token, the parser was switched to a comment mode where > characters were actually allowed within the comment. If the -- sequence was encountered, it changed to a different mode where the > would end the comment.
In fact, it appears to have been a toggle switch between those two modes, so something like <!-- >>>>> -- xyzzy -- >>>>> --> was possible, but putting a > where the xyzzy would end the comment.
XML, for one, didn't adopt this behaviour and HTML has now modified it to follow the "don't use -- within comments at all" rule, the reason being that hardly anyone knew that the comments behaved in the SGML way, causing some pain :-)
I am working on a processor that parts texts into blocks with marks:
LOREM IPSUM SED AMED
will be parsed like:
{word:1}LOREM{/word:1}{space:2}
{word:3}IPSUM{/word:3}{space:4}
{word:5}SED{/word:5}{space:6}
{word:7}AMED{/word:7}
But I dont want to use "{word}" etc, because it causes processor down, because it is an string again... I need to mark like these:
\E002\0001 LOREM \E003\0001 \E004\0002
\E002\0003 IPSUM \E003\0004 \E004\0005
\E002\0006 SED \E003\0006 \E004\0007
\E002\0008 AMED \E003\0008
First \E002 means element type number, its last bit represent element's close. So element number increments with +2.
Second \0001 means element index for stacking.
I am just used \E002 irrelevantly for this example.
But \0001 also using in Unicode Range, and this leads me to where I start again...
So which unicode range can I use? \ff0000? or how can I solve this?
Thanks!
The Unicode Consortium thought of this. There is a range of Unicode code points that are meant to never represent a displayable character, but meta-codes instead:
Noncharacters are code points that are permanently reserved and will never have characters
assigned to them.
...
Tag characters were intended to support a general scheme for the internal tagging of text
streams in the absence of other mechanisms, such as markup languages. The use of tag
characters for language tagging is deprecated.
(http://www.unicode.org/versions/Unicode9.0.0/ch23.pdf)
You should be able to use regular control characters as "private" tags, because these should never occur in proper strings. This would be the range from U+0000 to U+001F, excluding tab (U+0009), the common "returns" (U+000A and U+000D), and, for safety, U+0000 itself (some libraries do not like Null characters in the middle of strings).
Non-characters
Noncharacters are code points that are permanently reserved in the Unicode Standard for
internal use. They are not recommended for use in open interchange of Unicode text data.
You can use U+FEFF (which is currently officially defined as Not-A-Character), or U+FFFE and U+FFFF. There are several more "officially not-a-characters" defined, and you can be fairly sure they would not occur in regular text strings.
A few random sequences with predefined definitions, and so highly unlikely to occur in plain text strings are:
Specials: U+FFF0–U+FFF8
The nine unassigned Unicode code points in the range U+FFF0..U+FFF8 are reserved for
special character definitions.
Annotation Characters: U+FFF9–U+FFFB
An interlinear annotation consists of annotating text that is related to a sequence of annotated
characters. For all regular editing and text-processing algorithms, the annotated characters
are treated as part of the text stream. The annotating text is also part of the content,
but for all or some text processing, it does not form part of the main text stream.
Tag Characters: U+E0000–U+E007F
This block encodes a set of 95 special-use tag characters to enable the spelling out of ASCIIbased
string tags using characters that can be strictly separated from ordinary text content
characters in Unicode.
(all quotations from the chapter as above)
Staying within conventions, you can also use U+2028 (line separator) and/or U+2029 paragraph separator.
Technically, your use of U+E000–U+F8FF (the "Private Use Area") is okay-ish, because these code points only can define an unambiguous character in combination with a certain font. However, it is possible these codes may pop up if you get your plain text from a source where the font was included.
As for how to encode this into your strings: it doesn't really matter if the numerical code immediately following your private tag marker is a valid Unicode character or not. If you see one of your own tag markers, then the value immediately following is always your own private sequence number.
As you see, there are lots of possibilities. I guess the most important criterium is whether you want to use other functions on these strings. If you create a string that is technically invalid Unicode (for instance, because it includes not-a-character values), some external functions may choose to fail to work on them, or silently remove the bad values. In such a case, you'd need to rigorously stick to a system in which you only use 'valid' code points.
I was (re)reading Joel's great article on Unicode and came across this paragraph, which I didn't quite understand:
For example, you could encode the Unicode string for Hello (U+0048
U+0065 U+006C U+006C U+006F) in ASCII, or the old OEM Greek Encoding,
or the Hebrew ANSI Encoding, or any of several hundred encodings that
have been invented so far, with one catch: some of the letters might
not show up! If there's no equivalent for the Unicode code point
you're trying to represent in the encoding you're trying to represent
it in, you usually get a little question mark: ? or, if you're really
good, a box. Which did you get? -> �
Why is there a question mark, and what does he mean by "or, if you're really good, a box"? And what character is he trying to display?
There is a question mark because the encoding process recognizes that the encoding can't support the character, and substitutes a question mark instead. By "if you're really good," he means, "if you have a newer browser and proper font support," you'll get a fancier substitution character, a box.
In Joel's case, he isn't trying to display a real character, he literally included the Unicode replacement character, U+FFFD REPLACEMENT CHARACTER.
It’s a rather confusing paragraph, and I don’t really know what the author is trying to say. Anyway, different browsers (and other programs) have different ways of handling problems with characters. A question mark “?” may appear in place of a character for which there is no glyph in the font(s) being used, so that it effectively says “I cannot display the character.” Browsers may alternatively use a small rectangle, or some other indicator, for the same purpose.
But the “�” symbol is REPLACEMENT CHARACTER that is normally used to indicate data error, e.g. when character data has been converted from some encoding to Unicode and it has contained some character that cannot be represented in Unicode. Browsers often use “�” in display for a related purpose: to indicate that character data is malformed, containing bytes that do not constitute a character, in the character encoding being applied. This often happens when data in some encoding is being handled as if it were in some other encoding.
So “�” does not really mean “unknown character”, still less “undisplayable character”. Rather, it means “not a character”.
A question mark appears when a byte sequence in the raw data does not match the data's character set so it cannot be decoded properly. That happens if the data is malformed, if the data's charset is explicitally stated incorrectly in the HTTP headers or the HTML itself, the charset is guessed incorrectly by the browser when other information is missing, or the user's browser settings override the data's charset with an incompatible charset.
A box appears when a decoded character does not exist in the font that is being used to display the data.
Just what it says - some browsers show "a weird character" or a question mark for characters outside of the current known character set. It's their "hey, I don't know what this is" character. Get an old version of Netscape, paste some text form Microsoft Word which is using smart quotes, and you'll get question marks.
http://blog.salientdigital.com/2009/06/06/special-characters-showing-up-as-a-question-mark-inside-of-a-black-diamond/ has a decent explanation.