This question already has answers here:
Stop Excel from automatically converting certain text values to dates
(37 answers)
Closed 6 years ago.
I am having a challenge with how MSOffice deals with number formats.
While I believe this is similar root cause to: Stop Excel from automatically converting certain text values to dates
It is different as this is not a date format and this involves both Excel and PowerPoint with VBA.
I have data that I am pulling out of a dB into CSV files and I am doing a .Replace on certain text markers (e.g. ##ReplaceText##) in a PPT template. (There is a good post on the site on how to do this I can't seem to locate now)
There is one field I need to deal with which is tracking a metric, this field is text in my dB, but it can contain special characters - specifically $ and %.
e.g. I could see the following values in the CSV file:
"increase market share","1234","$10","28%"
I want VBA to treat this all as text, so the % and $ characters are maintained...but... Excel reads the data as a number and keeps the $ or % sign. PowerPoint removes the $ or % sign and converts 28% to 0.28 and $10 to 10.
Per the above question, adding "=""28%""" to the .csv in Excel, will give me that exact literal text in PowerPoint.
Adding a preceding space or ' character works in forcing Excel to read the data as text string. But PowerPoint ignores it and behaves same as above. Eg 28% to 0.28.
I tried using FORMAT as below, but because the data is variable, I don't know which case to apply.
sCurrentText = Format(sCurrentText, "$#")
or
sCurrentText = Format(sCurrentText, "0.0%")
If statements don't work because the $ or % are not present in what VBA sees (e.g the $ or % character is already gone)
If sCurrentText Like "*$*" Then or If sCurrentText Like "*%" Then
So my question is how do I force VBA to take what is in the CSV file as text and ignore processing $ or % as special characters and just maintain them in the CSV?
You didn't specify what exactly you want to do with the data in the CSV file, but I've assumed you're trying to open the file in VBA.
If you are opening the CSV file using OpenText (as below) then Excel will automatically parse the data in the format it sees fit. eg:
Workbooks.OpenText fileName:="directory", DataType:=xlDelimited, Comma:=True
You can use a different method to open the CSV file if you want VBA to handle the data as just text which you can use as you see fit.
Sub OpenCSVFile()
Dim ff As Long, iRow As Long, iCol As Long
Dim FilePath As String
Dim FileBuffer As String 'Entire CSV file as one string
Dim LineSeparatedFile() As String 'Array of data separated into lines
Dim LineData() As String 'Array of comma separated values for that line
ff = FreeFile
Open FilePath For Binary Access Read As #ff
FileBuffer = Space$(LOF(ff))
Get #ff, , FileBuffer
Close #ff
LineSeparatedFile = Split(txtBuffer, vbCrLf)
For iRow = 0 To UBound(LineSeparatedFile)
LineData = Split(LineSeparatedFile(i), ",")
For iCol = 0 To UBound(LineData)
'Code to do something with each entry.
'Eg. print to cell as text
ThisWorkbook.Sheets(1).Cells(iRow + 1, iCol + 1).NumberFormat = "#"
ThisWorkbook.Sheets(1).Cells(iRow + 1, iCol + 1).Value = LineData(iCol)
Next iCol
Next iRow
End Sub
Related
How do I get excel to recognize timestamps as timestamps rather than strings?
If I do a find and replace on "/" with "/" it fixes it on most files:
Cells.Replace What:="/", Replacement:="/", LookAt:=xlPart, SearchOrder :=xlByRows, MatchCase:=False, SearchFormat:=False, ReplaceFormat:=False
I have a chunk of code that checks if it's still in the wrong format by converting to "Comma" format and checking if the cell contains any "/" characters, then a break line that triggers in that instance to alert me that I need to manually do the find and replace on this file. If I stop the Macro when it fails and run it manually (Crtl+h, Enter), then it works and I can restart the macro to finish the standardisation. I need a way of automating this.
I have >2000 .csv files of a similar but not identical format. Each one contains ~350 variables, each with it's own timestamp and data column. I've written some code that formats it into a usable format. The original csv has the timestamps in "DD/MM/YYYY hh:mm:ss" format as is my computer and Excel default.
Excel seemingly randomly decides it can't recognise around a quarter of the files timestamps and instead interprets them as strings. This can be corrected by clicking into the cell and then clicking out of the cell, then excel recognises it as a timestamp. I need the timestamps recognised so that I can interpolate values into a standard sampling frequency as Excel can't interpolate using values it interprets as strings.
There are often well over 100k timestamps per file, so doing this manually isn't an option.
I've tried using SendKeys. The problem with that seems to be that it opens the find and replace dialogue for the VBA script editor, not for the excel sheet.
I've tried shifting focus before by calling:
Windows(windowname).Activate
ActiveWorkbook.Application.SendKeys("^h")
I've also tried:
Windows(windowname).Application.SendKeys("^h")
Which both result in the find and replace being called on the VBA script editor.
I have no shortcut to start the Macro.
I've tried Matlab, but it can't deal with the header on the file or the columns populated with text. I'd like to retain all the data.
I have used the Macro recorder to record me doing the find and replace which results in:
Sub Fixer()
'
' Fixer Macro
'
'
Selection.Replace What:="/", Replacement:="/", LookAt:=xlPart, _
SearchOrder:=xlByRows, MatchCase:=False, SearchFormat:=False, _
ReplaceFormat:=False
End Sub
But this doesn't work when you run it on the same file.
I expect it to convert the string "DD/MM/YYYY hh:mm:ss" format into a date-time format that I can then convert into decimal format which I can then use for interpolating the values into a usable format. Instead I get no error message, and nothing happens.
An example line of date timestamps from the raw CSV is:
"31/03/2019 14:55:57,1.0000000149,31/03/2019 14:55:57,14.6,31/03/2019 14:55:57,57.86,31/03/2019 14:55:57,0.175000000000068"
So the timestamp "31/03/2019 14:55:57" I want converting into "43555.62218750000"
I could use a script to deconstruct the string, calculate the decimal equivalent, and overwrite the cell, but this will take a prohibitively long time.
First import the date/time field into Excel as Text. In my demo case I use column A. Then run:
Sub TextToValue()
Dim Kolumn As String, N As Long, i As Long
Dim d As Double, dt As Date, tm As Date
Dim v As Variant
Kolumn = "A"
N = Cells(Rows.Count, Kolumn).End(xlUp).Row
For i = 1 To N
v = Cells(i, Kolumn).Text
If InStr(1, v, "/") > 0 Then
arr = Split(v, " ")
brr = Split(arr(0), "/")
dt = DateSerial(brr(2), brr(1), brr(0))
tm = TimeValue(arr(1))
d = CDbl(dt + tm)
Cells(i, Kolumn).Clear
Cells(i, Kolumn).Value = d
End If
Next i
End Sub
Before:
And after:
You need to back up a step.
Your problem is common and is caused by OPENing a csv file where the date format (DMY) in this case is in a different format than your Windows Regional Setting on the your computer.
So even the input that appears to convert properly, will not be as the Day and Month will be exchanged from what you might expect.
Arguably the "best" fix for this issue, assuming you cannot alter the csv file, will be to IMPORT the file instead.
Depending on how you do the IMPORT, and that is Excel version dependent, you will have the opportunity, at the time of import, to define the date format of the incoming data, so it will be properly converted by Excel.
In Power Query you can specify to have PQ do the date conversion according to a specified locale. In earlier versions of Excel, a Text-to-columns wizard will open allowing you to specify DMY for the format of the csv file. I would suggest using PQ as it can easily handle the time portion, whereas with the older wizard you'll need to split the two, and then add them back together.
I have a Text/CSV File of more than 10,000,000 Rows and 3 Columns.
Columns Names: ClientName, CLientMobile, ClientData
ClientData is in Hex format.
Presently I am doing:
Splitting the File in multiple parts of 900,000 rows each
Opening Each File - Say File 1
Pasting the below stated Function as Macro (Macro for Hex2Text)
Public Function HexToText(Text As Range) As String
Dim i As Integer
Dim DummyStr As String
For i = 1 To Len(Text) Step 2
DummyStr = DummyStr & Chr(Val("&H" & (Mid(Text, i, 2))))
DoEvents
Next i
HexToText = DummyStr
End Function
Converting Each Hex Value on Column "ClientData" in Readable Text by using above Function "Hex2Text"
Saving the Sheet.
Issues Faced:
I have to split all such big files in 900,000 row limit due to Excel limitations
It takes lot of time for calculations to run when I copy past formulae for Hex2Text for all 900,000 rows for Hex Values conversion in "ClientData"
Solution Desired:
Is there any other software that I can use for the same purpose to avoid spitting and avoid huge time wasted in Excel calculations for Hext2Text conversion.
Any simple solution/idea's will be welcome.
I am trying to process a large amount of data in VBA (in excel).
I have thousands of lines of strings that look like this:
LABEL_PERCENT XXX.XX% LABEL_DATE mm/dd/yy
I have used split to process line-by-line (so I am looking at an individual string as defined above). All of the lines have that exact formatting. For each line, I'd like to extract the percentage, and date, for populating a spreadsheet. How do I process the string in VBA, such that I can extract the values into two new variables?
You are already using Split()? This function is how you could extract the four values, splitting on the spaces:
Dim str As String
Dim splitted As Variant
str = "LABEL_PERCENT XXX.XX% LABEL_DATE mm/dd/yy"
splitted = Split(str, " ")
Debug.Print splitted(1) 'XXX.XX%
splitted(3) will give you the date. You then might want to parse the values as a percentage and date.
I have csv string (utf-8) obtained via a http download.
Depending on the situation the data in the string could contain a different number of columns, but each individual time a string is processed it will contain the same number of columns and be contiguous. (the data will be even).
The string could contain any number of rows.
The first row will always be the headings.
String fields will be encased in double quotes and could contain commas, quotes and newlines.
quotes and double quotes inside a string are escaped by doubling so "" and ''
In other words this is a well formed csv format. Excel through it's standard file open mechanism has no problem formatting this data.
However I want to avoid saving to a file and then opening the csv as I will need to process the output in some cases, or even merge with existing data on a worksheet.
(Added the following information via edit)
The Excel Application will be distributed to various destinations and I want to avoid if possible potential permissions issues, seems that writing nothing to disk is a good way to do that
I am thinking something like the following pseudo:
rows = split(csvString, vbCrLf) 'wont work due to newlines inside string fields?
FOREACH rows as row
fields = split(row, ',') 'wont work due to commas in string fields?
ENDFOR
Obviously that cant handle the fields containing special tokens.
What is a solid way of parsing this data?
Thanks
EDIT 13/10/2012 Data Samples
csv as it would appear in notepad (note not all line breaks will be \r\n some could be \n)
LanguageID,AssetID,String,TypeID,Gender
3,50820,"A string of natural language",3,0
3,50819,"Complex text, with comma, "", '' and new line
all being valid",3,0
3,50818,"Some more language",3,0
The same csv in Excel 2010 - opened from shell (double click - no extra options)
If you don't mind putting the data in your workbook: You could use a blank worksheet, add the data in 1 column, then call TextToColumns. Then if you want to get the data back as an array just load it from the UsedRange of the worksheet.
'Dim myArray 'Uncomment line if storing data to array.
'Assumes cvsString is already defined
'Used Temp as sheet for processing
With Sheets("Temp")
.Cells.Delete
.Cells(1, 1) = cvsString
.Cells(1, 1).TextToColumns Destination:=Cells(1, 1), DataType:=xlDelimited, _
TextQualifier:=xlDoubleQuote, ConsecutiveDelimiter:=False, Tab:=False, _
Semicolon:=False, Comma:=True, Space:=False, Other:=False
'myArray = .UsedRange 'Uncomment line if storing data to array
End With
I can think of three possibilities:
Use Regular Expressions to process the text. There are plenty of examples available on SO and via google for separating strings like this.
Use the power of Excel: save the text to a temp file, open into a temp sheet and read the data off the sheet. Delete the file and sheet when done.
Use ADO to query the data. Save the string to a temp file and run a query on that to return the fields you want.
To offer any more specific advice I would need samples of input data and expected output
I have a 2200+ page text file. It is delivered from a customer through a data exchange to us with asterisks to separate values and tildes (~) to denote the end of a row. The file is sent to me as a text file in Word. Most rows are split in two (1 row covers a full line and part of a second line). I transfer segments (10 page chunks) of it at a time into Excel where, unfortunately, any zeroes that occur at the end of a row get discarded in the "text to columns" procedure. So, I eyeball every "long" row to insure that zeroes were not lost and manually re-enter any that were.
Here is a small bit of sample data:
SDQ EA 92 1551 378 1601 151 1603 157 1604 83
The "SDQ, EA, and 92" are irrelevant (artifacts of data transmission). I want to use Excel and/or VBA to select 1551, 1601, 1603, and 1604 (these are store numbers) so that I can copy those values, and transpose paste them vertically. I will then go back and copy 378, 151, 157, and 83 (sales values) so that I can transpose paste them next to the store numbers. The next two rows of data contain the same store numbers but give the corresponding dollar values. I will only need to copy the dollar values so they can be transpose pasted vertically next to unit values (e.g. 378, 151, 157, and 83).
Just being able to put my cursor on the first cell of interest in the row and run a macro to copy every other cell would speed up my work tremendously. I have tried using ActiveCell and Offset references to select a range to copy, but have not been successful. Does any have any suggestions for me? Thanks in advance for the help.
It's hard to give a complete answer without more information about the file.
I think if your input data is 2200+ pages long, it's unlikely that opening it with the default excel opening functions is the way to go. Especially since Excel has maximum number of rows and columns. If the file is a text file (.txt) I would suggest opening it with VBA and reading each line, one at a time, and processing the data.
Here's an example to get you started. Just keep in mind that this is transposing each row of text into columns of data, so you will quickly fill all the columns of excel long before you run thru 2200 pages of text. But it's just an example.
Sub getData()
dFile = FreeFile
sFile = "c:\code\test.txt"
Open sFile For Input As #dFile
c = 1
'keep doing this until end of file
Do While Not EOF(dFile)
'read line into dataLine
Input #dFile, dataLine
' break up line into words based on spaces
j = Split(dataLine, " ")
jLength = UBound(j)
If jLength > 2 Then
r = 1
'ignore first 3 words
'and get every other word
'transpose rows of text into columns
For word = 3 To jLength Step 2
Cells(r, c) = j(word)
r = r + 1
Next word
End If
c = c + 1
Loop
Close #Data
End Sub