Conversion from hexadecimal string to Double yields wrong results - excel

I am trying to convert 14 bit hex numbers to decimal.
I have this VBA code.
Option Explicit
Public Function HexadecimalToDecimal(HexValue As String) As Double
Dim ModifiedHexValue As String
ModifiedHexValue = Replace(HexValue, "0x", "&H")
HexadecimalToDecimal = CDec(ModifiedHexValue)
End Function
With numbers like this to convert to decimal
0x047B1142591E80
0x044A81325A1E80
0x047B7542591E80
I keep getting random results across large amounts of data. Sometimes spot on other times the numbers are off by 6 or 2.

Try changing the return type of the function from Double to Variant. Double has only about 15 decimal digits of precision, so can't, for example, capture the value 1261213964639872 (which has 16 digits) exactly. The closest it can get is 1261213964639870. By changing the return type to Variant, the full precision returned by CDec will be preserved. You can't use a Decimal return type, because VBA for some reason does not support this.

The problem isn't with VBA. Excel cells can only hold 15 digits in number format. So the "number" 1234567891234567 will always display 1234567891234560. This can be avoided by converting items to text AND/OR changing the cell format to text.
But this doesn't always work.
The only surefire way to make sure it will retain all digits is to append something onto the string that isn't a number.
This code will append an apostrophe before the number, but return the entire string.
Public Function HexadecimalToDecimal(HexValue As String) As String
Dim ModifiedHexValue As String
ModifiedHexValue = Replace(HexValue, "0x", "&H")
HexadecimalToDecimal = "'" & CDec(ModifiedHexValue)
End Function
Unfortunately, not a perfect solution.

Related

how to dim a variable that has both numbers and letters?

As I am new to Excel VBA, how to classify a variable that has both letters and numbers? Like for instance "f0rmym8" do I Dim As Long or Dim As String or something else?
Thanks!
As per the documentation the string type variable is the only one capable of holding characters. (with the exclusion of variant, being able to hold an array of strings). All other data types are practically purely numeric, including Date and Boolean in VBA, with Date being effectively a Double formatted as a date, and a Boolean being -1 (true) or 0 (false).
In summary whenever you need to hold any kind of ASCII character that is not numeric, you will need to use the string data type, regardless of whether there are also numeric characters present. A numeric data type is always incapable of holding any type of text.
A String can hold any group of characters, consider:
Sub catscradle()
Dim Letters As String, MixedStuff As String, JustNumerals As String
Dim NothingAtAll As String
Letters = "ABC"
MixedStuff = "1A2B3C"
JustNumerals = "0076941"
NothingAtAll = ""
End Sub

VBA Strings without Dollar Signs

I'm learning about VBA, and I've noticed an odd quirk when VBA interacts with adding strings. When you take the two versions of code (Change the commented parts with the swap here commented parts) one outputs a string with a dollar sign, and one does without. Does anybody know if this is a bug or is planned to be upgraded?
Option Explicit
Sub CalcCost()
Dim curSalesPrice As Currency
Dim curTotalCost As Currency
Dim sngSalesTax As Single
Dim strMessage As String
curSalesPrice = 35
sngSalesTax = 0.085
Range("A1:B8").ClearContents
Range("A1").Value = "The cost of the calculator"
Range("A4").Value = "Price"
Range("B4").Value = curSalesPrice
Range("A5").Value = "SalesTax"
Range("A6").Value = "Cost"
Range("B5").Value = curSalesPrice * sngSalesTax
'curTotalCost = curSalesPrice + (curSalesPrice * sngSalesTax)
curTotalCost = Format(curSalesPrice + (curSalesPrice * sngSalesTax), "Currency") 'swap here
'strMessage = "The calculator total is " & Format(curTotalCost, "Currency")
strMessage = "The calculator total is " & curTotalCost 'swap here
Range("A8").Value = strMessage
Range("B6").Value = curTotalCost
End Sub
Format is a VBA standard library function defined in the VBA.Strings module; it returns a String representation of the expression it's given, formatted as specified: it makes no sense to do this:
Dim foo As Currency ' a numeric data type...
foo = Format(123, "Currency") ' ...assigned to a string representation of a numeric value
But it makes complete sense here:
Dim msg As String
msg = Format(123, "Currency")
Now, the value of a cell is distinct from its text representation. It's not because you see $123.00 in a cell that the value of that cell is $123.00 (a String); that's the cell's Text, but its Value can very well be 123 (a Double) and its NumberFormat be $#0.00.
You want to use numeric data types to perform operations, and use Format only when you need to make these numeric values "pretty" for display. Avoid making arithmetic operations on strings: while that may work, it also may fail, depending on how the string is formatted, and the system locale: VBA needs to make implicit type conversions to carry out such operations, and implicit conversions need to make a number of (sometimes wrong) assumptions.
When writing numeric values to worksheet cells, write the numeric values, not a string representation of them (same for dates. especially dates, actually). Instead of Format-ing the values, specify a format string for Range.NumberFormat in the cells that need to be formatted. That way Excel will still understand the numeric values as such, and can still correctly perform e.g. SUM operations.
The code is working exactly as specified and intended.

EXCEL VBA: Scientific Notation in a Userform Textbox

I am developing a userform in EXCEL using VBA. One of the default values for a user input in the form is 1E-12. I need to display this in the userform. However, when I initialize the textbox in the form, EXCEL immediately changes the representation of the value to 0.000000000001 (Both in the editor and in the userform when I check it). Computationally, of course, this is not an issue, however, it is a little tough to read for the user (and takes up to much space in the form to display the whole number). Is there a way to force EXCEL to show this value in scientific notation in the userform textbox?
Thanks!
Dan
You can use the VBA.Strings.Format$ function to control the string representation of numbers.
Specifying a "Scientific" format string will sort-of work, but if you need to control the level of precision, you'll need a more elaborate format string. You can make a FormatScientific function to abstract away that complexity:
Public Function FormatScientific(ByVal value as Double, Optional ByVal precision As Long = 6) As String
If precision < 1 Then precision = 1
Dim formatString As String
formatString = "0." & String(precision - 1, "0") & "#e-#"
FormatScientific = Format$(value, formatString)
End Function
Here I made the precision optional parameter be 6 if unspecified; adjust as you see fit.
Now you can invoke FormatScientific(0.0000123456789) and get 1.234568e-5 with a consistent number of significant digits, regardless of what you give it, whereas a "Scientific" format string only gives you 1.23E-05, which loses quite a lot of the significant information.
(adapted from parts of this code)
tbNum.Text = Format(Sheet1.Cells(1, 1).Value, "Scientific")
Takes value in the cell and converts it to scientific notation.

Excel : Find only Hexa decimals from 1 cell

I'm a newbie on Excel.
So I have a list of some names ending with Hexa decimals. And some names, that doesn't have any.
My mission is to see only those names with Hexa decimals. (Mabye somehow filter them out)
Column:
BFAXSPOINTDEVBAUHOFLAN2AD
BFAXSQLBAUHOFLAN207
BFAXSQLDEVBAUHOFLAN27A
BFREPDEVBAUHOFLAN258
BFREPORTINGBAUHOFLAN20B
COBALTSEA02900
COBALTSEAVHOST900
DIRECTO8000
DIRECTO9000
DIRECTODCDIRECTOLA009
DYNAMAEBSSISE006
SURVEYEBSSISE006
KVMSRV00",
KVMSRV01",
KVMSRV02",
ASR
CACTI
DBSYNC",
DTV
and so on...
The Function HEX2DEC will help you achieve what you want - it attempts to convert a number as a hexidecimal, into a decimal. If it is not a valid Hex input, it will produce an error.
The key is understanding how many digits you expect your decimal to be - is it the last 5 characters; the last 10; etc. Also note that there is a risk that random text / numbers will be seen as hexidecimal when really that's not what it represents [but that's a problem with the question as you have laid it out; going solely based on the text provided, all we can see is whether a particular cell creates a valid Hexidecimal].
The full formula would look like this[assuming your data starts in A1, and that your Hexidecimal numbers are expected to be 6 characters long, this goes in B1 and is copied down]:
=ISERROR(HEX2DEC(RIGHT(A1,6)))
This takes the 6 rightmost characters of a cell, and attempts to convert it from Hex to Decimal. If it fails, it will produce TRUE [because of ISERROR]; if it succeeds, it will produce FALSE.
Then simply filter on your column to see the subset of results you care about.
Consider the following UDF:
Public Function EndsInHex(r As Range) As Boolean
Dim s As String, CH As String
s = r(1).Text
CH = Right(s, 1)
If CH Like "[A-F]" Or CH Like "[0-9]" Then
EndsInHex = True
Else
EndsInHex = False
End If
End Function
For the string to end in a hex, the last character must be a hex.

When putting string that into Excel spreadsheet it puts to Scientific notation or rounds up

I am putting a string into excel. The string is often only numeric digits but can have alpha characters or hypens etc.
When I don't set the number format or set it like this
(Where xlSheet(0) is Excel.Worksheet)
xlSheet(0).Columns("N:N").EntireColumn.Columns.NumberFormat = "#"
It outputs in scientific notation.
When I use this code:
xlSheet(0).Columns("N:N").EntireColumn.Columns.NumberFormat = "0"
It rounds up the number to the nearest 100,000 so that the last five digits are 0's when they shouldn't be.
Should be: 1539648751235678942
But is: 1539648751235600000
The cells that have a hyphen or a letter aren't affected and work fine.
Any help would be greatly appreciated.
EDIT:
I add the data like this:
I loop through and put in xlSheet(0).Cells(i, 14) = rs!value_number
Where rs is my ADODB.Recordset
EDIT2: Herbert Sitz got it by adding an apostrophe before the text! Thanks everyone.
I think problem is that the number you're trying to enter can't be accommodated exactly by Excel. Excel has limitations on what numbers it display/represent because of the way numbers are stored internally. In Excel's case numbers are limited to 15 digit precision (see http://office.microsoft.com/en-us/excel-help/excel-specifications-and-limits-HP010073849.aspx ), which is not enough to represent your number.
You can enter the number as a string ("152..42") and all digits will be displayed, but you won't be able to perform exact mathematical operations with it.
For numbers, Excel can only handle 15 significant digits.
If you want to store a number that is more than 15 digits long without losing data, you have to store the data as text.
Doing what you've been doing will resolve the issue:
You can do either of the following to add your numbers as text:
xlSheet(0).Cells(i, 14).Numberformat = "#"
xlSheet(0).Cells(i, 14) = rs!value_number
Or
xlSheet(0).Cells(i, 14) = "'" & rs!value_number

Resources