| BigDecimal issue [message #545682] |
Thu, 01 March 2012 11:57  |
 |
aucrun
Messages: 114
Registered: February 2011
|
Senior Member |
|
|
Hi,
I'm having an problem using BigDecimal Library for Javascript.
Imagine I have the following value:
3.438470
This evil number is giving me the headaches, because I 'm not able to round it as I need.
Using the BigDecimal library I am not able to round it to 3 decimal places correctly. It allways rounds it to 3.438 instead of the 3.439 value it should return.
There are two major rounding modes I'm using:
- ROUND_UP
- ROUND_HALF_UP
The first solves this particular problem but gives me another one, since, as described in its documentation (here)
it will never decrements the original value, and it the original number changes to 3.438440 it would return 3.439 also.
The second gives me allways 3.438 because, as written in its documentation it only sees if the marginal numbers to the right of the Decimal Places are bigger or smaller than 0.5
this is wrong, because, in this particular case, 3.438470 should be:
3.43847
3.4385
and finally 3.439
Anyone has an idea of how to solve this?
thakns!
|
|
|
|
| Re: BigDecimal issue [message #546747 is a reply to message #545682] |
Thu, 08 March 2012 09:16  |
|
aucrun
Messages: 114
Registered: February 2011
|
Senior Member |
|
|
Hi again,
I've actually found out a way of doing it.
We can iterate the SetPlaces n times, accordingly with the decimal places to the right of the required decimal place.
In the example:
Value to Round - 3.438447
Rounded Value from C# (decimal datatype) - 3.438
Rounded Value from BigDecimal (Round_Half_Up) - 3.438
Regular Round Value from Excel - 3.438
Required Result - 3.439
Using the BigDecimal setScale we can do this:
var bd_number= new BigDecimal("3.438447");
var result = bd_number.setScale(4, BigDecimal.prototype.ROUND_HALF_UP).setScale(3, BigDecimal.prototype.ROUND_HALF_UP);
I think, that the main problem is that, with huge numbers the amount of iterations may consume a lot of processor resources!
It works for me, since I only need to look for the first 6 decimal places to the right of zero! and round only by the first 3!
|
|
|
|
] 
Blog
Search
Help
Members
Register
Login
Home
