Do you like a forecast that says, “There is a chance for life-threatening storm surge and your home may be inundated by floodwaters and you may not survive” or a forecast that says, “There is a chance for up to 12ft of storm surge with this hurricane and that will inundate your entire home. If you live in a one-story building it is not survivable”?
At the American Meteorological Society Annual Meeting in Denver, I was lucky enough to get a chance to listen to drop in on a talk from Abby Lee Bitterman, from the Oklahoma University Institute for Public Policy Research and Analysis.
She broke down the use of numbers and words in forecasts dedicated for public consumption. Officially, the title of the talk was, “Numbers or Words?: How Perceptions of Probability Comprehension Influence the Communication Pathway for Weather Forecasting Products..”
The presentation was a shortened version of the scholarly article, “There’s a Chance of What? Assessing Numeracy Skills of Forecasters, Emergency Managers, and Publics to Improve TC Product Uncertainty Communication, IDSS, and Training.”
While the paper and the talk are meant more for tropical cyclone, I think it could be applicable – at times – across the board.
Quick Background
People – forecasters, public, doctors, dogs, cats, and probably even Aliens from another solar system – all struggle with understanding fractions and percentages. Certain people are better than others, but everyone struggles.
On top of that, fractions and percentages are real, actual numbers but when use in a forecast can be made out to describe non-real, non-actual things… Like the percent change for rain. Conversely, if I told you that you had 1/3 of an inch of rain, you could look in a container and see, hold, taste, and smell that rain.
A percent chance for rain isn’t very tactile. It is a bit more abstract. And thus, it is up for a bit of interpretation (but not a ton)
People who go to school and take a bunch of math classes end up with a better feel for what a percent chance of something happening means, but even then it isn’t completely cut-and-dry.
Bitterman’s Presentation
The talk looked at how forecasters use number to communicate the forecast to you, the public, through local Emergency Management officials (like during a hurricane or major flooding event) and how you, the public, consume that information.
A snippet of the abstract reads: “The use of probabilistic information in forecast products is becoming increasingly more common in the weather enterprise, as the National Weather Service aims to further establish the use of probabilistic information in communicating with partners and members of the public. The capacity to understand probabilistic information differs across the adult population, with some people finding probability incredibly intuitive, others finding it impossible to grasp and everyone else falling on a continuum in between. The following study investigates how emergency manager (EM) and forecaster perceptions of public probability comprehension impact forecast communication strategies.”
And the authors of the study didn’t just say that the public struggles, they ahve the numbers to back it up.
While receiving forecast information might feel pretty straight-forward, the research showed that the steps in place between the NWS forecaster making a forecast and you, the public, receiving that forecast are more than just one step in some cases.
The communication of that forecast is multi-layered.
Sure, some times things go directly from NWS forecaster to you, the human in teh public, but other times it has to be relayed though local Emergency Management folks.
And when the NWS Forecaster discusses the forecast with the local EM official, the forecast can lose some of its numerical data. Why? About 1/4 of forecasters surveyed through that you, the public, didn’t understand numbers in the forecast well enough.
Half of the forecasters surveyed said they needed better tools to describe the forecast numerically to the EMs, and thus you.
So… What? I’m not an EM, Nick, Where are you going with this?
Right.
According to the research, numeracy (the ability to work with an understand numbers – both simple and complex) has a larger impact on comprehension of probability information. That is true across the board. So, for you, the public, it is important to take some time to sit and think about what numbers mean when they are apart of a forecast.
It matters. Research suggests that members of the public with lower levels of numeracy “often misinterpreted products, misjudged risk, and incorrectly assessed the protective actions that are appropriate given a forecast.”
So it becomes very, very important that when you hear “There is a 15% risk for severe weather” or “The Forecast Cone means there is a 66% chance the center of the storm is in this area” that you stop and think about what that actually mean.
And if you don’t know, aren’t sure, or just want a refresher… ask!
The one place that I would love to hear from YOU is that the research also suggested that brief forecast discussions with extra info – but no explicit numbers – significantly improved public interpretation of products. And those same explanations that included “numeric information” were better at communicating the forecast according to the people surveyed.
This is where you come in!
So, do you like forecasts with both words explaining the forecast and the numbers telling you there is a 15% chance for severe weather or a 70% chance for tropical storm force wind? Does that help you make a decision about what protective action to take? Let me know!
Authors of the research said that finding out that people do like both an explanation and the numbers “contradicts the tendency of forecasters and EMs to translate numeric information to less precise words and phrases when explaining forecast products to members of the public.”
I like both the “numeric information” and the words explaining the forecast.