Prevent mammoth numerical errors by forcing SQL Server to treat values as decimals



Request Information

Want more information? Fill out the form below and we'll contact you shortly.

Name

Phone

Email

Subject

Message




Date : 7/29/2010

What's 2 divided by 3? Anyone for 0 as the answer? Unfortunately, that's what you may get if you're not fully aware of how SQL Server treats numbers in queries.

For example, try running the following query:

SELECT 2/3

And the winner is ...

Zero. The reason SQL Server returns 0 should be pretty obvious in this example. Both 2 and 3 are integers; therefore, SQL Server assumes the result should be an integer. Hence, it rounds down to the nearest integer, which is 0.

Unfortunately, figuring out that this issue was the culprit may not be so easy with a more complicated expression inside a complex application. Instead, your end result may be something quite bizarre. Worse still, nobody may even question the critically wrong result.

A quick and dirty way to fix this problem is to force SQL Server to think in decimals by adding trailing zeros, like this:

SELECT 2.0/3.0

In fact, you only have to do this with either one of the two operands, and SQL Server will still return the same value: 666666. (You could call this math problem "the Devil's division.") Add several more zeros after the decimal, and SQL Server will reserve more memory and thus return more sixes after the decimal.

One advantage of adding trailing zeros is that it lets you avoid bogging down your queries with too many queries. When generating SQL, your application code can add the zeros automatically. (Just make sure it never adds zeros when there's no decimal!)

Of course, the alternative is to use the CAST() or CONVERT() functions to convert to data types such as decimal, float, or real, like so:

SELECT cast(2 as decimal)
/cast(3 as decimal)

Curiously, when you use the CAST() function for both numbers, SQL Server returns 6666666666666666666, whereas using it for only one of them still gives you just 6 sixes. So just be aware that SQL Server calculates precision a little differently depending on whether you use trailing zeros or functions.

 


Previous News: 7/20/2010

Avoid misconceptions about SQL injection attacks

Avoid the hassle of escaping quotes with user-defined quotes

Create standalone software packages with the Microsoft Access Runtime

Never hit the wrong database again in the Query window

Remove a sender from your Outlook junk e-mail list (2003/2007)

Split your screen using only the keyboard (Word 2000/2002/2003/2004/2007)

Same Day

Browse for audio files on FindSounds.com

Create recurring appointments in your Microsoft Outlook calendar (2000/2002/2003/2007)

Keep unnecessary files out of sight, but close at hand (Excel 2000/2002/2003/2004/2007)

Protect your Web servers with URLScan

Validate radio buttons with a quick JavaScript

Next News: 8/10/2010

Avoid misconceptions about SQL injection attacks

Back up your login password to avoid losing it altogether

Define a reusable attributeGroup in your XML Schema

Detect SQL and bind variables with extended auditing (10g)

Display multiple Outlook Windows at once (2000/2002/2003/2007)

Divide administrative responsibilities for best security

How to get users to create more secure passwords

Let DateDiff() determine if two dates are in the same month (VBScript 2.0)

Make a scrapbook of slides on your desktop (PowerPoint 2000/2002/2003)

Select objects hidden by other objects (PowerPoint 2000/2002/2003/2004/2007)

Simple shortcuts to help you edit DataGrid data (VB 2003)

Test out font sizes quickly (Word 2000/2002/2003/2004/2007)

Share |
Since 1982, New Horizons Computer Learning Centers has grown to become the largest independent IT training company worldwide. We offer more courses, at more times and in more locations than any other computer and business training company. Businesses and individuals can choose from hundreds of course offerings, delivered by experts, and offered in hundreds of convenient locations around the world.


Albany, NY
(518) 452-6444
10 Airline Drive, Ste 101
Albany, NY 12205
Charlotte, NC
(704) 522-9747
9140 Arrowpoint Blvd, Ste 400
Charlotte, NC 28273
Harrisburg, PA
5095 Ritter Road, Ste 114
Mechanicsburg, PA 17055
Lancaster, PA
(717) 791-0200
1020 New Holland Ave
Lancaster, PA 17601