Reading medium large .dat file with Excel VBA - excel

Question

Your support is really appreciated!
I am receiving a .dat file from a measuring tool, which is found hard to get in to excel.
I would like to do it without power query as well.
I do this in steps:
Step 1; convert dat file to "csv/txt" by removing duplicate spaces and replacing spaces with ";", also replacing "." with ",".
I would like to keep this format as several other tools tends to use similar format.
And from this I thought it would be fairly ok to import it, however...
First row of 11000 rows of .dat file:
1 1 -0.4200 -0.0550 0.1420 173 174 181 56.3 55.5 59.3 87 84 95 0.778 0 0 0
first row of the converted file, all rows below looks good as well.
1;1;-0,4260;-0,1500;0,0990;171;168;176;55,5;53,8;57,6;96;83;82;4,794;0;0;0
if I import this file with power query it seems ok.
Step 2:
When importing it with the code below, following occurs on line 660
from txt file
1;660;-1,0210;-0,0340;0,0470;169;164;176;54,6;51,2;57,2;15;96;63;0,782;0;0;0
from excel:
Debuging the shows following:
file:
format of the cell is "Numbers" and not "geeral" as for other numbers.
This seems to occure now and then, and typically when the number goes above -1,xx.
Code is found online, and is fairly quick.
I suspect that something happens when putting the two-dimensional variant array into the sheet
Dim Data As Variant 'Array for the file values
.
.
.
.
With Sheets(parSheetName)
'Delete any old content
.cells.ClearContents
'A range gets the same dimensions as the array
'and the array values are inserted in one operation.
.cells(4, 1).Resize(UBound(Data, 1), UBound(Data, 2)) = Data
End With
End If
Option Explicit
'**************************************************************
' Imports CSV to sheet, following the generated numbers will be placed in a table.
'**************************************************************
Public Sub copyDataFromCsvFileToSheet(parFileName As String, _
parDelimiter As String, parSheetName As String)
Dim Data As Variant 'Array for the file values
Dim I As Long
Dim J As Long
Dim prt As String
'Function call - the file is read into the array
Data = getDataFromFile(parFileName, parDelimiter)
'If the array isn't empty it is inserted into
'the sheet in one swift operation.
If Not isArrayEmpty(Data) Then
'If you want to operate directly on the array,
'you can leave out the following lines.
ActiveWorkbook.Worksheets.Add(After:=Sheets(Sheets.Count)).Name = parSheetName
'For I = 1 To 1000 'UBound(Data, 1)
'For J = 1 To 18 'UBound(Data, 2)
''prt = Data(I, J)
''Debug.Print prt
''ThisWorkbook.Worksheets(parSheetName).cells(I, J) = Data(I, J)
'Next J
'Next I
'Debug.Print "done"
'End If
With Sheets(parSheetName)
'Delete any old content
.cells.ClearContents
'A range gets the same dimensions as the array
'and the array values are inserted in one operation.
.cells(4, 1).Resize(UBound(Data, 1), UBound(Data, 2)) = Data
End With
End If
'Call sbCreatTable(parSheetName)
End Sub
'**************************************************************
Private Function isArrayEmpty(parArray As Variant) As Boolean
'Returns False if not an array or a dynamic array
'that hasn't been initialised (ReDim) or
'deleted (Erase).
If IsArray(parArray) = False Then isArrayEmpty = True
On Error Resume Next
If UBound(parArray) < LBound(parArray) Then
isArrayEmpty = True
Exit Function
Else
isArrayEmpty = False
End If
End Function
Private Function getDataFromFile(parFileName As String, _
parDelimiter As String, _
Optional parExcludeCharacter As String = "") As Variant
'parFileName is the delimited file (csv, txt ...)
'parDelimiter is the separator, e.g. semicolon.
'The function returns an empty array, if the file
'is empty or cannot be opened.
'Number of columns is based on the line with most
'columns and not the first line.
'parExcludeCharacter: Some csv files have strings in
'quotations marks ("ABC"), and if parExcludeCharacter = """"
'quotation marks are removed.
Dim locLinesList() As Variant 'Array
Dim locData As Variant 'Array
Dim I As Long 'Counter
Dim J As Long 'Counter
Dim locNumRows As Long 'Nb of rows
Dim locNumCols As Long 'Nb of columns
Dim fso As Variant 'File system object
Dim ts As Variant 'File variable
Const REDIM_STEP = 10000 'Constant
'If this fails you need to reference Microsoft Scripting Runtime.
'You select this in "Tools" (VBA editor menu).
Set fso = CreateObject("Scripting.FileSystemObject")
On Error GoTo error_open_file
'Sets ts = the file
Set ts = fso.OpenTextFile(parFileName)
On Error GoTo unhandled_error
'Initialise the array
ReDim locLinesList(1 To 1) As Variant
I = 0
'Loops through the file, counts the number of lines (rows)
'and finds the highest number of columns.
Do While Not ts.AtEndOfStream
'If the row number Mod 10000 = 0
'we redimension the array.
If I Mod REDIM_STEP = 0 Then
ReDim Preserve locLinesList _
(1 To UBound(locLinesList, 1) + REDIM_STEP) As Variant
End If
locLinesList(I + 1) = Split(ts.ReadLine, parDelimiter)
J = UBound(locLinesList(I + 1), 1) 'Nb of columns in present row
'If the number of columns is then highest so far.
'the new number is saved.
If locNumCols < J Then locNumCols = J
I = I + 1
Loop
ts.Close 'Close file
locNumRows = I
'If number of rows is zero
If locNumRows = 0 Then Exit Function
ReDim locData(1 To locNumRows, 1 To locNumCols + 1) As Variant
'Copies the file values into an array.
'If parExcludeCharacter has a value,
'the characters are removed.
If parExcludeCharacter <> "" Then
For I = 1 To locNumRows
For J = 0 To UBound(locLinesList(I), 1)
If Left(locLinesList(I)(J), 1) = parExcludeCharacter Then
If Right(locLinesList(I)(J), 1) = parExcludeCharacter Then
locLinesList(I)(J) = _
Mid(locLinesList(I)(J), 2, Len(locLinesList(I)(J)) - 2)
Else
locLinesList(I)(J) = _
Right(locLinesList(I)(J), Len(locLinesList(I)(J)) - 1)
End If
ElseIf Right(locLinesList(I)(J), 1) = parExcludeCharacter Then
locLinesList(I)(J) = _
Left(locLinesList(I)(J), Len(locLinesList(I)(J)) - 1)
End If
locData(I, J + 1) = locLinesList(I)(J)
Next J
Next I
Else
For I = 1 To locNumRows
For J = 0 To UBound(locLinesList(I), 1)
locData(I, J + 1) = locLinesList(I)(J)
Next J
Next I
End If
getDataFromFile = locData
Exit Function
error_open_file: 'Returns empty Variant
unhandled_error: 'Returns empty Variant
End Function
Due to mentioned several measuring tools, the power query is un suited, and the control is better when using the ole way of doing it.

Answer 1

Solution:
Setting the variant to decimal when building the array
CDec(locLinesList(I)(J))
Thanks for your responce!