I'm having issues writing strings to binary in Lua. There is an existing example and I tried modifying it. Take a look:
function StringToBinary()
local file = io.open("file.bin", "wb")
local t = {}
local u = {}
local str = "Hello World"
file:write("string len = " ..#str ..'\n')
math.randomseed(os.time())
for i=1, #str do
t[i] = string.byte(str[i])
file:write(t[i].." ");
end
file:write("\n")
for i=1, #str do
u[i] = math.random(0,255)
file:write(u[i].." ");
end
file:write("\n"..string.char(unpack(t)))
file:write("\n"..string.char(unpack(u)))
file:close()
end
file:write(t[i].." ") and file:write(u[i].." ") write both tables with integer value. However with my last two writes: unpack(t) displays the original text, while unpack(u) displays the binaries.
It's probably string.byte(str[i]) that is mistaken. What should I replace it with? Am I missing something?
t[i] = string.byte(str[i])
is wrong, it should be:
t[i] = string.byte(str, i)
Related
How'd I change everything that it matches in the string without changing the non matches?
local a = "\" Hello World! I want to replace this with a bytecoded version of this!\" but not this!"
for i in string.gmatch(a, "\".*\"") do
print(i)
end
For example I want [["Hello World!" Don't Replace this!]] to [["\72\101\108\108\111\32\87\111\114\108\100\33" Don't Replace this!]]
Your question is a little bit tricky because it could involve:
Lua patterns
string.gsub function
Access to string's bytes with string.byte
String concatenations with table.concat
First things first, if you need to implement Lua patterns, please know that there is a very handy Lua syntax which is very appropriate for dealing with quoted strings. With this syntax, instead of opening/closing a string with a double-quote, you do it with the characters [[ and ]]. The key difference is that between these markers, you don't have to escape the quoted strings anymore!
String = [["Hello World!" Don't Replace this!]]
Then, we need to build the proper Lua pattern, a possibility could be to match a double-quote (") and then match all the characters which are not a double-quote ("), this gives us the following pattern:
[["([^"]+)"]]
| **** |
| \-> the expression to match
| |
quote quote
Then if we study the function string.gsub, we can learn that the function can call a callback when a pattern is matched, the matched string will be replaced by the return value of the callback.
function ConvertToByteString (MatchedString)
local ByteStrings = {}
local Len = #MatchedString
ByteStrings[#ByteStrings+1] = [[\"]]
for Index = 1, Len do
local Byte = MatchedString:byte(Index)
ByteStrings[#ByteStrings+1] = string.format([[\%d]], Byte)
end
ByteStrings[#ByteStrings+1] = [[\"]]
return table.concat(ByteStrings)
end
In this function, we iterate through all the characters of the matched string. Then for each of the characters, we extract its byte value with the function string.byte and convert it to a string using the string.format function. We put this string in a temporary array that we will concatenate at the end of the function.
The function to concatenate the sub-strings into a larger string is table.concat. This is a very convenient function which could be used as follow:
> table.concat({ [[\10]], [[\11]], [[\12]] })
\10\11\12
The remaining thing we need to do is to test this outstanding function:
> String = [["Hello World!" Don't Replace this!]]
> NewString = String:gsub([["([^"]+)"]], ConvertToByteString)
> NewString
\"\72\101\108\108\111\32\87\111\114\108\100\33\" Don't Replace this!
Edit: I got some remarks regarding code performances, I personally don't focus much on performances, I focus on getting the code correct & simple. In order to address the performance question, I wrote a micro-benchmark to compare the versions:
function SOLUTION_DarkWiiPlayer (String)
local result = String:gsub('"[^"]*"', function(str)
return str:gsub('[^"]', function(char)
return "\\" .. char:byte()
end)
end)
return result
end
function SOLUTION_Robert (String)
local function ConvertToByteString (MatchedString)
local ByteStrings = {}
local Len = #MatchedString
ByteStrings[#ByteStrings+1] = [[\"]]
for Index = 1, Len do
local Byte = MatchedString:byte(Index)
ByteStrings[#ByteStrings+1] = string.format([[\%d]], Byte)
end
ByteStrings[#ByteStrings+1] = [[\"]]
return table.concat(ByteStrings)
end
local Result = String:gsub([["([^"]+)"]], ConvertToByteString)
return Result
end
function SOLUTION_Piglet (String)
return String:gsub('%b""' , function (match)
local ret = ""
for _,v in ipairs{match:byte(1, -1)} do
ret = ret .. string.format("\\%d", v)
end
return ret
end)
end
function SOLUTION_Renshaw (String)
local function convert(str)
local byte_str = ""
for i = 1, #str do
byte_str = byte_str .. "\\" .. tostring(string.byte(str, i))
end
return byte_str
end
local Result = string.gsub(String, "\"(.*)\"", function(matched_str)
return "\"" .. convert(matched_str) .. "\""
end)
return Result
end
String = "\"Hello World!\" Don't Replace this!"
print("INITIAL REQUIREMENT FROM OP ", [[\"\72\101\108\108\111\32\87\111\114\108\100\33\" Don't Replace this!]])
print("TEST SOLUTION_Robert: ", SOLUTION_Robert(String))
print("TEST SOLUTION_DarkWiiPlayer:", SOLUTION_DarkWiiPlayer(String))
print("TEST SOLUTION_Piglet: ", SOLUTION_Piglet(String))
print("TEST SOLUTION_Renshaw: ", SOLUTION_Renshaw(String))
The results show that only one answer fulfill 100% of OP's requirements. The other answers doesn't handle the first and ending double-quotes " properly.
INITIAL REQUIREMENT FROM OP \"\72\101\108\108\111\32\87\111\114\108\100\33\" Don't Replace this!
TEST SOLUTION_Robert: \"\72\101\108\108\111\32\87\111\114\108\100\33\" Don't Replace this!
TEST SOLUTION_DarkWiiPlayer: "\72\101\108\108\111\32\87\111\114\108\100\33" Don't Replace this!
TEST SOLUTION_Piglet: \34\72\101\108\108\111\32\87\111\114\108\100\33\34 Don't Replace this! 1
TEST SOLUTION_Renshaw: "\72\101\108\108\111\32\87\111\114\108\100\33" Don't Replace this!
To finalize this post, one could dive a little deeper and check the code performances with a micro-benchmark which could be copy/paste directly in a Lua interpreter.
function SOLUTION_DarkWiiPlayer (String)
local result = String:gsub('"[^"]*"', function(str)
return str:gsub('[^"]', function(char)
return "\\" .. char:byte()
end)
end)
return result
end
function SOLUTION_Robert (String)
local function ConvertToByteString (MatchedString)
local ByteStrings = {}
local Len = #MatchedString
ByteStrings[#ByteStrings+1] = [[\"]]
for Index = 1, Len do
local Byte = MatchedString:byte(Index)
ByteStrings[#ByteStrings+1] = string.format([[\%d]], Byte)
end
ByteStrings[#ByteStrings+1] = [[\"]]
return table.concat(ByteStrings)
end
local Result = String:gsub([["([^"]+)"]], ConvertToByteString)
return Result
end
function SOLUTION_Piglet (String)
return String:gsub('%b""' , function (match)
local ret = ""
for _,v in ipairs{match:byte(1, -1)} do
ret = ret .. string.format("\\%d", v)
end
return ret
end)
end
function SOLUTION_Renshaw (String)
local function convert(str)
local byte_str = ""
for i = 1, #str do
byte_str = byte_str .. "\\" .. tostring(string.byte(str, i))
end
return byte_str
end
local Result = string.gsub(String, "\"(.*)\"", function(matched_str)
return "\"" .. convert(matched_str) .. "\""
end)
return Result
end
---
--- Micro-benchmark environment
---
COUNT = 600000
function TEST_Function (Name, Function, String, Count)
local TimerStart = os.clock()
for Index = 1, Count do
Function(String)
end
local ElapsedSeconds = (os.clock() - TimerStart)
print(string.format("[%25.25s] %f sec", Name, ElapsedSeconds))
end
String = "\"Hello World!\" Don't Replace this!"
TEST_Function("SOLUTION_DarkWiiPlayer", SOLUTION_DarkWiiPlayer, String, COUNT)
TEST_Function("SOLUTION_Robert", SOLUTION_Robert, String, COUNT)
TEST_Function("SOLUTION_Piglet", SOLUTION_Piglet, String, COUNT)
TEST_Function("SOLUTION_Renshaw", SOLUTION_Renshaw, String, COUNT)
The results shows that #DarkWiiPlayer's answer is the fastest one.
[ SOLUTION_DarkWiiPlayer] 6.363000 sec
[ SOLUTION_Robert] 9.605000 sec
[ SOLUTION_Piglet] 7.943000 sec
[ SOLUTION_Renshaw] 8.875000 sec
local a = "\"Hello World!\" but not this!"
print(a:gsub('"[^"]*"', function(str)
return str:gsub('[^"]', function(char)
return "\\" .. char:byte()
end)
end))
you need string.gsub.
local a = "\"Hello World!\" Don't Replace this!"
local function convert(str)
local byte_str = ""
for i = 1, #str do
byte_str = byte_str .. "\\" .. tostring(string.byte(str, i))
end
return byte_str
end
a = string.gsub(a, "\"(.*)\"", function(matched_str)
return "\"" .. convert(matched_str) .. "\""
end)
print(a)
local a = "\" Hello World! I want to replace this with a bytecoded version of this!\" but not this!"
print((a:gsub('%b""' , function (match)
local ret = ""
for _,v in ipairs{match:byte(1, -1)} do
ret = ret .. string.format("\\%d", v)
end
return ret
end)))
How can I add a method to the string table and modify self inside it ?
Basically, I'm trying to mimic the behaviour of the io.StringIO.read method in python, which reads n char in the string and returns them, modifying the string by "consuming" it.
I tried this:
function string.read(str, n)
to_return = str:sub(1, n)
str = str:sub(n + 1)
return to_return
end
local foo = "heyfoobarhello"
print(string.read(foo, 3))
print(foo)
Output is:
hey
heyfoobarhello
I expected the second line to be only foobarhello.
How can I achieve this ?
To mimic Python's io.StringIO class, you must make an object that stores both the underlying string and the current position within that string. Reading from an IO stream normally does not modify the underlying data.
local StringIO_mt = {
read = function(self, n)
n = n or #self.buffer - self.position + 1
local result = self.buffer:sub(self.position, self.position + n - 1)
self.position = self.position + n
return result
end,
}
StringIO_mt.__index = StringIO_mt
local function StringIO(buffer)
local o = {buffer = buffer, position = 1}
setmetatable(o, StringIO_mt)
return o
end
local foo = StringIO"heyfoobarhello"
print(foo:read(3))
print(foo:read())
Output:
hey
foobarhello
I don't recommend adding this class or method to Lua's string library, because the object has to be more complex than just a string.
You can add methods to the datatype string independently from the string table.
Short example that shows that the string methods even work if string table gets deleted...
string=nil
return _VERSION:upper():sub(1,3)
-- Returning: LUA
So you can add a method...
-- read.lua
local read = function(self, n1, n2)
return self:sub(n1, n2)
end
getmetatable(_VERSION).__index.read=read
return read
...for all strings.
( Not only _VERSION )
And use it...
do require('read') print(_VERSION:read(1,3):upper()) end
-- Print out: LUA
I am trying to separate the string data in an HTTP protocol in wireshark using lua and I am not having success finding the end of the string, this is what I currently have
HTTP_protocol = Proto("ourHTTP", "HTTPProtocol")
first =ProtoField.string("HTTP_protocol.first", "first", base.ASCII)
second =ProtoField.string("HTTP_protocol.second", "second", base.ASCII)
HTTP_protocol.fields = {first}
function HTTP_protocol.dissector(buffer, pinfo, tree)
length = buffer:len()
if length ==0 then return end
pinfo.cols.protocol = HTTP_protocol.name
local subtree = tree:add(HTTP_protocol, buffer(), "HTTPProtocol data ")
local string_length
for i = 0, length - 1, 1 do
if (buffer(i,1):uint() == '\r') then
string_length = i - 0
break
end
end
subtree:add(first, buffer(0,string_length))
end
porttable = DissectorTable.get("tcp.port")
porttable:add(80, HTTP_protocol)
i have tried searching for '\r', '\0' and '\n' but no matter what I still get all the strings inputed as one. Is there something I am doing wrong?
You can use 0x0D instead. That's the ASCII code for \r. So it will end up as
if (buffer(i,1):uint() == 0x0D) then
In Wireshark:
How would I attempt this?
I'm trying to create something that would remove all quotes (" ") in a Lua file but I have had no luck so far. But it might be because im a newbie at Lua.
I'm using this from GitHub.
function from_base64(to_decode)
local padded = to_decode:gsub("%s", "")
local unpadded = padded:gsub("=", "")
local bit_pattern = ''
local decoded = ''
for i = 1, string.len(unpadded) do
local char = string.sub(to_decode, i, i)
local offset, _ = string.find(index_table, char)
if offset == nil then
error("Invalid character '" .. char .. "' found.")
end
bit_pattern = bit_pattern .. string.sub(to_binary(offset-1), 3)
end
for i = 1, string.len(bit_pattern), 8 do
local byte = string.sub(bit_pattern, i, i+7)
decoded = decoded .. string.char(from_binary(byte))
end
local padding_length = padded:len()-unpadded:len()
if (padding_length == 1 or padding_length == 2) then
decoded = decoded:sub(1,-2)
end
return decoded
end
I'm trying to create something that would remove all quotes (" ") in a Lua file
-- read contents of file into memory
local file = io.open(filename)
local text = file:read('*a')
file:close()
-- remove all double-quotes from the contents
text = text:gsub('"','')
-- write contents back to the file
local file = io.open(filename, 'w+')
local text = file:write(text)
file:close()
I have a fairly large text file and am trying to search for a particular term so that i can start a process after that point, but this doesn't seem to be working for me:
fileID = fopen(resfile,'r');
line = 0;
while 1
tline = fgetl(fileID);
line = line + 1;
if ischar(tline)
startRow = strfind(tline, 'OptimetricsResult');
if isfinite(startRow) == 1;
break
end
end
end
The answer I get is 9, but my text file:
$begin '$base_index$'
$begin 'properties'
all_levels=000000000000
time(year=000000002013, month=000000000006, day=000000000020, hour=000000000008, min=000000000033, sec=000000000033)
version=000000000000
$end 'properties'
$begin '$base_index$'
$index$(pos=000000492036, lin=000000009689, lvl=000000000000)
$end '$base_index$'
definitely doesn't have that in the first 9 rows?
If I ctrl+F the file, I know that OptimetricsResult only appears once, and that it's 6792 lines down
Any suggestions?
Thanks
I think your script somehow works, and you were just looking at the wrong variable. I assume that the answer you get is startRow = 9 and not line = 9. Check the variable line. By the way, note that you're not checking an End-of-File, so your while loop might run indefinitely the file doesn't contain your search string.
An alternative approach, (which is much simpler in my humble opinion) would be reading all lines at once (each one stored as a separate string) with textscan, and then applying regexp or strfind:
%// Read lines from input file
fid = fopen(filename, 'r');
C = textscan(fid, '%s', 'Delimiter', '\n');
fclose(fid);
%// Search a specific string and find all rows containing matches
C = strfind(C{1}, 'OptimetricsResult');
rows = find(~cellfun('isempty', C));
I can't reproduce your problem.
Are you sure you've properly closed the file before re-running this script? If not, the internal line counter in fgetl does not get reset, so you get false results. Just issue a fclose all on the MATLAB command prompt, and add a fclose(fileID); after the loop, and test again.
In any case, I suggest modifying your infinite-loop (with all sorts of pitfalls) to the following finite loop:
haystack = fopen(resfile,'r');
needle = 'OptimetricsResult';
line = 0;
found = false;
while ~feof(haystack)
tline = fgetl(haystack);
line = line + 1;
if ischar(tline) && ~isempty(strfind(tline, needle))
found = true;
break;
end
end
if ~found
line = NaN; end
fclose(fileID);
line
You could of course also leave the searching to more specialized tools, which come free with most operating systems:
haystack = 'resfile.txt';
needle = 'OptimetricsResult';
if ispc % Windows
[~,lines] = system(['find /n "' needle '" ' haystack]);
elseif isunix % Mac, Linux
[~,lines] = system(['grep -n "' needle '" ' haystack]);
else
error('Unknown operating system!');
end
You'd have to do a bit more parsing to extract the line number from C, but I trust this will be no issue.