Browsers and number of decimal places

Hi,

I have a question concerning how browsers handle decimal places when using percentage or em units. I read somewhere ages ago that browsers only count the first 6 digits, regardless of where the decimal point is i.e. 123456, 1.23456, 12.3456, 123.456, 1234.56, 12345.6, etc before dropping further numbers. Is this correct, or what is the best practice for this sort of thing?

Thanks.

Different browsers round in different ways - some round according to normal mathematical rules, others simply truncate the number. You should aim to ensure your design isn’t reliant on a potential 1px rounding error.

What kind of design feature do you have in mind where you need browsers to use 7 significant figures in their calculations?

I don’t have the need to do that (and it’s 6, not 7) but I have been dabbling with em-based designs, and I read ages ago that browsers deal with up to 6 digits (regardless of the decimal point position), and recently I read that for ems, browsers like to use a maximum of 3 decimal places i.e. 1.234 for em-based sizing. So it’s a bit confusing really.

Unless you are dealing with huge monitors capable of displaying millions of hundreds of millions of pixels across the screen instead of the thousands that most monitors can handle the 3rd through 6th decimal places in those numbers are going to be too small to make any difference whatsoever.

So in other words, I’ll be fine using a maximum of 2 or 3 decimal places where necessary. That clears a lot up. Thanks.

wait… I thought the question was about DIGITS.

In other words do UAs limit the # of digits given? 10.3333em of 103.333em? In which case that would be rather limiting. I could understand why someone might find something with 12 digits behind the the decimal point a bit ridiculous, but when its just 6 ( 3 infort and 3 in back , for example) it IS a bit limiting, regardless of monitor size.

remember ems and percentages are RATIOS. Also, god forbid if 48" 108ppi solution monitors suddenly become vogue. The VERY advantage to using em/% is to describe a formula rather than approximate a specific measurement.

Correct. :slight_smile:

On a typical design, you’ve only got about 1000 pixels across to play with. Whatever calculations you’re doing if you’re looking for an answer between 1 and 1000, any difference in the 6th digit is not going to register even at the sub-pixel level (and remember that browsers can’t address sub-pixels anyway), so I can see no situation where needing the precision of an extra digit would come into play.

If your design is that tightly-screwed that it needs a 7 sig.fig. number in its calculations, it’s going to be well and truly screwed by the different rounding algorithms that browsers have, regardless of the level of precision used in the calculation.

well what you are saying Steve is that there is little difference in 33333.3em and 3.33333em when in fact since the rounding occurs BY FRACTIONS of a pixel I would be much more concerned with the decimals. (any whole # is going to give you a whole number) so your problem will lie in the fractional part of your ratio.Hence it would have been nice if the figured had been allowed to 6 decimals rather than six significant digits

Well most people would have a very long scroll just to see the one letter at 33333.3em as that would be several thousand feet high and most people’s monitors are nowhere near that. You can easily drop three digits off the front of that number and still have letters that will require a lot of scrolling as they will still be several times as tall as most people’s monitors. So three significant digits is way overkill for current monitor sizes and six significant digits will easily handle monitors where the screen size is measured in miles.

Of course if your monitor is seceral miles high and you actually are using a 33333.3em font size then can you please post a photo of it so we can see the difficulty of getting fonts to display correctly when the individual letters are over a mile in height.

I think you slipped a decimal point or two. On a 100ppi screen, the usual default is 6em per in. Therefore 33333.3em=5555.5in=462.96ft, still too big for anything but government work, but not quite miles. :wink:

cheers,

gary

Looks like I must have. Not used to working with that stone age measurement system - did learn it back when I was young (how many chains make a furlong etc) but have been using the metric system for the past 30+ years and so didn’t notice that I’d slipped a decimal place or two.

Still it doesn’t significantly change the fact that 6 significant digits is more than will ever be needed for defining sizes on any screen.

Of course there’s also the fact that if the letters are over 400 feet high then a few inches of rounding would not be noticed (and with 6 significant digits the system can still round those letters to the nearest pixel so to get a few inches rounding error you’d need letters that were many miles high - where the few inches would be even less likely to be noticed).

It is true that Imperial/US length measures are outdated. Volume measures, though, are thoroughly modern compared to an now outdated metric based on a decimal system. The old is new again.

One mouthful, now called a tablespoon, is the base. Two mouthfuls is an ounce, and two ounces is a jack. Double again for a gill, cup, and then a pint. Each measure is double the previous; quart, pottle (bottle), gallon, peck, bushel, on up to a tun (2^16 mouthfuls). A tun of water weighs approximately a ton. Who’d a thunk it?

How about that! A computer ready binary system of measures from medieval times. :smiley:

cheers,

gary

:rofl::rofl::rofl::rofl::rofl:

One mouthful, now called a tablespoon, is the base. Two mouthfuls is an ounce, and two ounces is a jack. Double again for a gill, cup, and then a pint. Each measure is double the previous; quart, pottle (bottle), gallon, peck, bushel, on up to a tun (2^16 mouthfuls). A tun of water weighs approximately a ton. Who’d a thunk it?

I kinda thought you were joking, but hubby’s been brewing and they really do use a “mash tun” which I thought was just a goofy name for the container. It was originally an amount.

One thing to keep in mind with large em’s is that Opera and Konqueror (and possibly other browsers) have limits (in pixels) which means if you exceed those limits with em’s then you’ll get wrapping to the other side or other strange effects.

Why I stopped using -9999em and switched to -999em for off-screen placement. Opera and Konq had kittens. ErikJ said the limit was something like 32000px ?

Well if we’re going to talk measurements :smiley:

Imperial also makes more sense for construction and there’s a reason it’s still dominant in aviation… in terms of distance measurements it’s the only one to use 2, 3 and 4. At anything a foot or larger you can always divide by 2 or 3 without fractions.

Though the idea system would be base 60 like we use for time… since that nets you 2,3,4 and 5 without fractional results.

But hey, I’m the guy who still measures distance in chains (22 yards. 1 square chain == 1 acre), cubits. (18 inches) and have even resorted to nails (2.25") and digits (0.75") on occasion… and the other day was svetting at the grocery store over how you can’t get a half gallon of ice-cream anymore as they now only seem to come in 4 mutchkin size.

Really base ten is a horrible system by comparison.