How to keep special characters when renaming a file using AppleScript - excel

Question

I've got an AppleScript file to replace filenames which are supplied through a CSV file. While I've got the script to work, it has issues with encoding of the strings/filenames.
The finding of files works perfectly. Renaming it to something like Malmö results in a very weird encoded string.
The CSV originates from Microsoft Excel, and I suspect is not properly UTF-8 encoded. And now I'm stuck in how to handle the encoding properly. (or how to convert the encoding). As far as I know it has the default Excel encoding ISO 8859-1.
set theFile to (choose file with prompt "Select the CSV file")
set thePath to (choose folder with prompt "Select directory") as string
set theCSVData to paragraphs of ((read theFile))
set {oldTID, my text item delimiters} to {my text item delimiters, ";"}
repeat with thisLine in theCSVData
set {oldFileName, newFileName} to text items of thisLine
if length of oldFileName > 0 then
set oldFile to thePath & oldFileName
set newFile to newFileName
tell application "System Events"
if exists file oldFile then
set name of file oldFile to newFile
end if
end tell
end if
end repeat
So my question is, how do I fix the encoding issue, either by reading it properly or by encoding the file first (through applescript)

Answer 1

I really think the filenames are encoded using utf16LE, but I can be wrong. Anyhow, there is a utility you can access via Terminal, that is named iconv (man iconv), that you can experiment with, and maybe re-encode the filename with. If you receive the output from a do shell script as text, or unicode text, then you get back utf16, so it won't be corrupted after the conversion. (Before you use it as a filename).

Answer 2

Use:
set theCSVData to paragraphs of ((read theFile as «class utf8»))