(Note: This post originally contained a remarkably stupid error in an example. For some idiotic reason, I calculated as if a liter was a cubic meter. Which, duh, it isn’t. so I was off by a factor of 1000. Pathetic, I know. Thanks to the multiple readers who pointed it out!)
The other day, I got a question via email that involves significant figures. Sigfigs are really important in things that apply math to real-world measurements. But they’re poorly understood at best by most people. I’ve written about them before, but not in a while, and this question does have a somewhat different spin on it.
Here’s the email that I got:
Do you have strong credentials in math and/or science? I am looking for someone to give an expert opinion on what seems like a simple question that requires only a short answer.
Could the matter of significant figures be relevant to an estimate changing from 20 to less than 15? What if it were 20 billion and 13.7 billion?
If the context matters, in the 80s the age of the universe was given as probably 20 billion years, maybe more. After a number of changes it is now considered to be 13.7 billion years. I believe the change was due to distinct new discoveries, but I’ve been told it was simply a matter of increasing accuracy and I need to learn about significant figures. From what I know (or think I know?) of significant figures, they don’t really come into play in this case.
The subject of significant digits is near and dear to my heart. My father was a physicist who worked as an electrical engineer producing power circuitry for military and satellite applications. I’ve talked about him before: most of the math and science that I learned before college, I learned from him. One of his pet peeves was people screwing around with numbers in ways that made no sense. One of the most common ones of that involves significant digits. He used to get really angry at people who did things with calculators, and just read off all of the digits.
He used to get really upset when people did things like, say, measure a plate with a 6 inch diameter, and say that it had an are] of 28.27433375 square inches. That’s ridiculous! If you measured a plate’s diameter to within 1/16th of an inch, you can’t use that measurement to compute its area down to less than one billionth of a square inch!
Before we really look at how to answer the question that set this off, let’s start with a quick review of what significant figures are and why they matter.
When we’re doing science, a lot of what we’re doing involves working with measurements. Whether it’s cosmologists trying to measure the age of the universe, chemists trying to measure the energy produced by a reaction, or engineers trying to measure the strength of a metal rod, science involves measurements.
Measurements are limited by the accuracy of the way we take the measurement. In the real world, there’s no such thing as a perfect measurement: all measurements are approximations. Whatever method we chose for taking a measurement of something, the measurement is accurate only to within some margin.
If I measure a plate with a ruler, I’m limited by factors like how well I can align the ruler with the edge of the plate, by what units are marked on the ruler, and by how precisely the units are marked on the ruler.
Once I’ve taken a measurement and I want to use it for a calculation, the accuracy of anything I calculate is limited by the accuracy of the measurements: the accuracy of our measurements necessarily limits the accuracy of anything we can compute from those measurements.
For a trivial example: if I want to know the total mass of the water in a tank, I can start by saying that the mass of a liter of water is one kilogram. To figure out the mass of the total volume of water in the tank, I need to know its volume. Assuming that the tank edges are all perfect right angles, and that it’s uniform depth, I can measure the depth of the water, and the length and breadth of the tank, and use those to compute the volume.
Let’s say that the tank is 512 centimeters long, and 203 centimeters wide. I measure the depth – but that’s difficult, because the water moves. I come up with it being roughly 1 meter deep – so 100 centimeters.
The volume of the tank can be computed from those figures: 5.12 times 2.03 times 1.00, or 10,393.6 liters.
Can I really conclude that the volume of the tank is 10,393.6 liters? No. Because my measurement of the depth wasn’t accurate enough. It could easily have been anything from, say, 95 centimeters to 105 centimeters, so the actual volume could range between around 9900 liters and 11000 liters. From the accuracy of my measurements, claiming that I know the volume down to a milliliter is ridiculous, when my measurement of the depth was only accurate within a range of +/- 5 centimeters!
Ideally, I might want to know a strong estimate on the bounds of the accuracy of a computation based on measurements. I can compute that if I know the measurement error bounds on each error measurement, and I can track them through the computation and come up with a good estimate of the bounds – that’s basically what I did up above, to conclude that the volume of the tank was between 9,900 and 11,000 liters. The problem with that is that we often don’t really know the precise error bounds – so even our estimate of error is an imprecise figure! And even if we did know precise error bounds, the computation becomes much more difficult when you want to track error bounds through it. (And that’s not even considering the fact that our error bounds are only another measured estimate with its own error bounds!)
Significant figures are a simple statistical tool that we can use to determine a reasonable way of estimating how much accuracy we have in our measurements, and how much accuracy we can have at the end of a computation. It’s not perfect, but most of the time, it’s good enough, and it’s really easy.
The basic concept of significant figures is simple. You count how many digits of accuracy each measurement has. The result of the computation over the measurements is accurate to the smallest number of digits of any of the measurements used in the computation.
In the water tank example, we had three significant figures of accuracy on the length and width of the tank. But we only had one significant figure on the accuracy of the depth. So we can only have one significant figure in the accuracy of the volume. So we conclude that we can say it was around 10 liters, and we can’t really say anything more precise than that. The exact value likely falls somewhere within a bell curve centered around 10 liters.
Returning to the original question: can significant figures change an estimate of the age of the universe from 20 to 13.7?
Intuitively, it might seem like it shouldn’t: sigfigs are really an extension of the idea of rounding, and 13.7 rounded to one sigfig should round down to 10, not up to 20.
I can’t say anything about the specifics of the computations that produced the estimates of 20 and 13.7 billion years. I don’t know the specific measurements or computations that were involved in that estimate.
What I can do is just work through a simple exercise in computations with significant figures to see whether it’s possible that changing the number of significant digits in a measurement could produce a change from 20 to 13.7.
So, we’re looking at two different computations that are estimating the same quantity. The first, 20, has just one significant figure. The second, 13.7 has three significant digits. What that means is that for the original computation, one of the quantities was known only to one significant figure. We can’t say whether all of the elements of the computation were limited to one sigfig, but we know at least one of them was.
So if the change from 20 to 13.7 was caused by significant digits, it means that by increasing the precision of just one element of the computation, we could produce a large change in the computed value. Let’s make it simpler, and see if we can see what’s going on by just adding one significant digit to one measurement.
Again, to keep things simple, let’s imagine that we’re doing a really simple calculation. We’ll use just two measurements 
Initially, we’ll say that we measured the value of 
After a few years pass, and our ability to measure
Now we’ve got equal precision on both of the measurements – they’re now both 2 significant figures. So we can compute a new, better estimate by multiplying them together, and reducing the solution to 2 significant figures.
We multiply 8.2 by 1.7, giving us around 13.94. Reduced to 2 significant figures, that’s 14.
Adding one significant digit to just one of our measurements changed our estimate of the figure from 20 to 14.
Returning to the intuition: It seems like 14 vs 20 is a very big difference: it’s a 30 percent change from 20 to 14! Our intuition is that it’s too big a difference to be explained just by a tiny one-digit change in the precision of our measurements!
There’s two phenomena going on here that make it look so strange.
The first is that significant figures are an absolute error measurement. If I’m measuring something in inches, the difference between 15 and 20 inches is the same size error as the difference between 90 and 95 inches. If a measurement error changed a value from 90 to 84, we wouldn’t give it a second thought; but because it reduced 20 to 14, that seems worse, even though the absolute magnitude of the difference considered in the units that we’re measuring is exactly the same.
The second (and far more important one) is that a measurement of just one significant digit is a very imprecise measurement, and so any estimate that you produce from it is a very imprecise estimate. It seems like a big difference, and it is – but that’s to be expected when you try to compute a value from a very rough measurement. Off by one digit in the least significant position is usually not a big deal. But if there’s only one significant digit, then you’ve got very little precision: it’s saying that you can barely measure it. So of course adding precision is going to have a significant impact: you’re adding a lot of extra information in your increase in precision!
That is truly the best explanation I’ve ever read of the subject. Thanks. I will keep a link to this post handy!
JR
Good explanation, but your measurement of the volume of the water tank is off by a factor of 1000 – it’s about 10 cubic metres, which is 10000 litres, not 10.
Yup. That’s what I get for trying to write when I’m under the weather. I calculated it as if a liter was the weight of a cubic meter of water. I’ve corrected it, and put a note at the top of the post pointing out the error.
Thanks for catching that!
Hi,
I think your tank calculations are out by a factor of 1000. A tank 512 centimetres by 203 centimetres by 100 centimetres would hold 10393.6 litres
Dave
Corrected, with a note at the top of the post acknowledging the error.
Check the units of your water tank example, something’s gone horrible wrong there. A tank of 512cm by 203cm by 1m would holds about 10000 liter.
Corrected, thanks for letting me know. Really stupid error on my part!
Going back to the original question, then if 1 significant figure indicates “a very rough measurement… very little precision…that you can barely measure it.” Then it was a mistake to describe any age as “probably.” Of course, people are always going to want a simple figure rather than a range, but as I recall there were scientists who seemed sure the actual figure couldn’t be too far from 20 billion. You’re also assuming that the figure is derived from a mathematical equation, but depending on the mathematical equation, mightn’t even a slight imprecision possibly result in a very wrong answer? You used a simple multiplication in your example, but what if the error were in a key factor in a complex formula with large exponents involved?
As to some of the specifics of the actual case, here’s what a professional astronomer had to say:
“The 20 billion year age is the Hubble time, which was based upon the Hubble constant being 50 km/s/Mpc, which was the standard value from 1960 to the early 1990s. Cosmologists assumed no cosmological constant but gravitational deceleration from the mass of the universe, which would reduce the actual age a bit. Estimates of that factor resulted in an age of 16-18 billion years. Expressing that a different way, the age was 17 billion years, plus or minus a billion years. That age held sway for 30 years, and notice that it would exclude any value outside that range, say 13.8 billion years. All during this time, globular star clusters were known to be at least 15 billion years old, so that itself would exclude a universe younger than that. That was the problem 25 years ago when it was discovered that the Hubble constant was significantly more than 50 km/s/Mpc, closer to 100 km/s/Mpc, putting the Hubble time in the 10 billion year range. The assumed value now is close to 80 km/s/Mpc, which gives a Hubble time of 12.5 billion years. There was a bit of a crisis at that time, because this required that the age of the universe be less than the known age of globular clusters. This was resolved by reevaluating the age of globular star clusters. It was dark energy that pushed the age of the universe back up. So the history of the age of the universe over the past 30 years is more like 17, 10, 12.5, and 13.8 billion years.”
BTW, have you run into the one about “The Bible says Pi = 3”?
Have you even heard of Fermi estimations?
This might seem like a diversion, but it’s not.
You want a rough idea about something, and you don’t have great data. So you just try to figure out the order of magnitude: is it in the 10s, the 100s, the 1000s?
It’s a valuable technique for creating rough estimates.
What it demonstrates is that even lousy precision is valuable: it can tell you valuable things, and allow you to make meaningful inferences.
Someone came up with a way of calculating an estimate of the age of the universe using the data they had available. It was a good estimate.
When better data became available, it was revised.
That’s science: you do the best you can with the data you have today, with the full knowledge that tomorrow’s observations could prove that you were wrong.
When that happens, it’s not a tragedy. You add the new data to the sum of what you know, and keep working to understand what it tells you.
And yes, I’ve seen the pi=3 nonsense. I find it completely uninteresting. It’s just a stupid “gotcha” thing.
Yes, sometimes you have to start with a “first approximation” and all that. The context of the age thing, though, was that it was presented not as a rough estimate, but practically a done deal. And then I had someone telling me that significant figures alone explained this going from 20 to under 15, so it was no big deal if 20 was presented as not likely to change much. Compare this to, say, the “changes” in the value of the charge or mass of the electron or the attraction of gravity during the same time period. They may not be perfectly comparable, but to most people it is all “science” and when a scientist says the universe is about 20 billion years old, people are going to think that’s the same as if a scientist says that G is about 7×10 N⋅m²/kg². But the later, given more precision, is approximately 6.674×10 N⋅m²/kg². See the difference?
Of course, I wasn’t saying the situation was tragic, I was merely pointing out that scientists sometimes present things (to the general public at least) as something we “know” or are reasonably certain isn’t likely to change much, when they should keep in mind precisely as you say, “the full knowledge that tomorrow’s observations could prove that you were wrong” and allow that to be clear in their communications to people who all-too-often are ready to take any pronouncement by scientists as the most reliable truth there can be.
Nothing in science is ever a done deal. Science is always the best approximation given what we know today, subject to revision, correction and/or refutation given new data.
You’re playing the old game of religion versus science. Religion says X, Y, and Z – and it’s revelation from God, absolute and utter truth, forever unchanging. Science says A, B, and C – and it’s best approximation given on what we know today, subject to revision tomorrow.
And now you’re trying to play with weasel-words. You started off by asking if it was possible that the change in estimates of the age of the universe could be explained solely by significant figures. Based on your response, you were clearly hoping that it couldn’t.
Now, when it’s clear that it could, you’re shifting the goalposts. No, you weren’t ever really concerned about whether significant figures could be the cause of the change. Now, you say, your concern all along was about something else: it’s all about how scientists state things too strongly when they’re uncertain.
It’s a classic bad-faith game. The kind of rubbish that gives all of us who are religious a bad reputation.
Did I contradict you or take you to task over something? I just wanted to see how significant figures might be considered relevant to such a large change. Indeed, I didn’t think that it could, but to be clear, I gave some of the background of the actual case (including, “… I believe the change was due to distinct new discoveries, but I’ve been told it was simply a matter of increasing accuracy…”).
You gave an excellent lesson on significant figures, and I didn’t offer any criticism of it. I merely went back to the original question and supplied some further information I had obtained, so we could all see if the theoretical case applied to the specific historical case. Don’t you think that “how scientists state things too strongly when they’re uncertain” is implied in the question of whether it was a matter of simply increasing accuracy or large variations/uncertainty in measurements?
You made it clear there was a difference between the theoretical possibilities and the actual case:
Intuitively, it might seem like it shouldn’t: sigfigs are really an extension of the idea of rounding, and 13.7 rounded to one sigfig should round down to 10, not up to 20.
I can’t say anything about the specifics of the computations that produced the estimates of 20 and 13.7 billion years. I don’t know the specific measurements or computations that were involved in that estimate.
So, going by the values 20 billion and 13.7 billion themselves, you can’t get from one to the other by rounding according to significant figures, right?
BUT, as you point out, if there are other factors involved, such as the values being derived by a calculation using still other values, at least one of whose values changed by improved accuracy and significant figures, then you might end up with a very large change.
As it turns out, there is a calculation involved, but estimated measurements of one of the values varied between various studies by even more than the difference between 20 and 13.7 (“Until recently, the best estimates ranged from 65 km/sec/Megaparsec to 80 …” http://map.gsfc.nasa.gov/universe/uni_age.html)
Of course, there is a religious view that posits the universe was created a few thousand years ago (although it may have aged billions of years, time being relative and all), and I believe it by faith, but I didn’t bring that up. If I’d known you might bite my head off when I didn’t even say you were wrong, I wouldn’t have asked you anything in the first place.
I don’t understand what you (David) are really going after here. What seemed to be a simple question of curiosity turned into a sort of diatribe on those arrogant, possibly even evil scientists.
Isn’t it always obvious and implicit, in any scientific pronouncement, that future knowledge could change current conclusions? 20 billion years was NEVER presented as infallible by the cosmology community at large. Of *course* estimates will vary as precision is improved. I challenge you to find authoritative sources (/respected cosmologists or astronomers) saying “20 billion years is completely accurate, precise to a million years, will never change”.
The error was not overconfidence on the part of cosmology, but misinterpretation on yours. Or, I suspect, bad faith: usually, this kind of searching for something to shake a finger at is motivated by other, previous conclusions and beliefs.
(If I’ve come across too harsh, I apologize– I just think it’s a very weird thing you’re doing here and I can’t help but pattern-match it to similar experiences.)
That’s pretty much my reaction. We started with a question that’s interesting from a mathematical viewpoint, about whether adding significant digits alone are enough to account for a substantial change in an estimated value. That’s what I initially answered.
Once it was clear that yes, sigdigs are enough to account for that, suddenly we had a long-winded (if relatively good-natured) rant about how all those nasty scientists misrepresented and it’s all dishonest.
Meh.
I don’t know why you felt you needed to jump in (and jump on me) with this “What seemed to be a simple question of curiosity turned into a sort of diatribe on those arrogant, possibly even evil scientists.” Did you also forget that I stated as background to the question that my curiosity was related to the apparent confidence in the original dating and what was the actual cause of the change? And so if I explained further about that and provided more information I had found out about it, how does that constitute turning into something else, and in what way was it “a sort of diatribe,” and where did I say that the scientists were “arrogant, possibly even evil” (or as Mark said, “nasty”)? If I HAD used such terms, you’d be justified in talking about a “diatribe.” You talk about “pattern matching” — maybe you should be careful that doesn’t become pigeon-holing.
So, of *course* I am a creationist — does it take a creationist to think a finger should be shook at statements made by experts, for public consumption, that turned out to be so far off? You challenged me: “Of *course* estimates will vary as precision is improved. I challenge you to find authoritative sources (/respected cosmologists or astronomers) saying “20 billion years is completely accurate, precise to a million years, will never change”” But if you’ve actually read through my comments you’ll see this was NOT just a matter of precision improving, and it wasn’t a range of “a million years” but over 5 billion years, more than 25% off. Back then, they weren’t telling the public that it was a rough estimate, let alone a Fermi approximation. No, I’m not saying they were evil, nasty, or even arrogant, just incidentally overconfident.
I just took a test with a research company, one question about how many doctors there were in California and how many of them made house calls. I could have knocked myself out getting the exact numbers, but I wouldn’t have had a leg to stand on if I claimed that was enough and the asker shouldn’t complain if I ignored the context — the asker was thinking of making an “uber for housecalls” app, and a simple search turned up the fact that there were already a couple of established companies with just that sort of app. Rather an important bit of information for the asker. I’d appreciate it if everyone would take the context I gave into account, and try to not see me as fitting your expectations as some kind of ranting religious nut.
Oh — I see my suspicion was well-founded; of *course* you are a creationist! It doesn’t mean you are a bad person… But at least I am well-calibrated.
Much smaller, but there is still one little error remaining. In the paragraph where you reason about bounding the 1 meter measurement you state the upper volume bound as 1100 liters instead of 11000 liters. In the following paragraph you do get both bounds right.
And then a teeny tiny style nit. In the paragraph with the remaining error you don’t use thousands-commas but you do in the next paragraph.
“…6 inch diameter, and say that it had an area of 18.84955592148 square inches”
Careful! That’s a perimeter of 18.8… inches (or an area of 28.2… square inches).
I can calculate the age of the universe using CTMU conjectures.
Sorry for the delay in modding this; it got caught by the spam filter.
One issue that I had to advise my students of when I was tutoring was that you should only use significant figures for rounding the final answer of a problem before reporting it. Rounding off intermediate results can result in much larger errors in the computation of the final answer.
Another point is that your rule of thumb about the significant digits of the answer being the smallest number of significant digits of any input really only applies to computations that are all multiplication and division. Adding 123,456.7 (seven significant digits) and 0.1 (1 significant digit) gives you 123,456.8, not 100,000. Subtracting 13.7 (three significant digits) from 13.8 (also 3 significant digits) gives you 0.1 (one significant digit), not 0.100 (3 significant digits). And error bounds/significant digits in exponents are a whole other can of worms.
In my experience, using significant digits to express uncertainty is a very crude tool, which quickly leads to unacceptable loss of accuracy in but the most simple situations.
In fact, even in simple situations it can quickly lead you into losing accuracy without even noticing it. Let’s take an example of a couple of years ago, when some columnist (I cannot remember now who) was making fun of the ad for a SUV that was boasting a ground clearance (or something like that) of 35.43 inches. Obviously, the information carried by those 4 significant digits is not that that clearance is accurate up to one ten-thousandth, but simply that it is the result of unit conversion – it’s 90 cm. How accurate are those 90 cm? Who knows, but let’s play the significant digit game and assume that it is correct up to 1cm, so that the actual clearance lies in the interval (90 ± 0.5)cm. Suppose now that the reported clearance in inches would be similarly expressed up to two significant digits, i.e., 35 in. Now I take the reported 35 inches and want to convert them to cm: using the same rules I get 89 cm, which is outside the initial estimate of (90 ± 0.5)cm. To sum up, the result I get is wrong.
This very example shows two of the problems of using significant digits for quantifying uncertainty: 1) it is sensitive to number representation (i.e. measurement units and number base) and 2) it conflates data uncertainty with rounding errors.
To obviate problem number two it is common to hear the suggestion to round off to the expected number of significant digits only the final result, not the intermediate ones. The problem with this is that, apart from the classroom or school lab, all numbers you produce are going to be used by somebody else for their calculations, so, in fact, pretty much all numbers are “intermediate” results.
“The volume of the tank can be computed from those figures: 5.12 times 2.03 times 1.00, or 10,393.6 liters.”
5.12m * 2.03m * 1.00m = 10.3936m3
1ltr = 1000cm3 ( 10cm * 10cm * 10cm )
1000cm3 = 0.001m3
10.3936m3 / 0.001m3 = 10,393.6ltr
I think it would be simpler if you used mathematical (computerized) notation along with units.
“So we conclude that we can say it was around 10 liters, and we can’t really say anything more precise than that. The exact value likely falls somewhere within a bell curve centered around 10 liters.”
This is confusing, where did the 10 liters come in from? Do you mean, approximately 10,000ltrs, or have I missed something here?
If you had stuck with cms to do all the calculations, then introduced the concept of a litre, that would have made things clearer for me.
I notice your note at the top, and the comments about an error, so I am left wondering a bit like Alice in Wonderland.