How not to report on pseudoscience

A great example of false balance and incorrect reporting of fringe pseudoscientific claims cropped up recently in the Sunday Herald. In this story, Wet, cold forecast for 2010by Rebecca Lewis, we encounter some rather odd reporting about New Zealand’s self proclaimed weather expert, Ken Ring. Lewis begins her article with the following passage:

 “Ken Ring, a long-term forecaster with unorthodox methods but a surprisingly accurate track record, has predicted next year’s weather to be “disappointing”, with wet and cooler summer months, followed by a winter that lasts a month longer than this year, with record-breaking cold snaps.

For background, Ken Ring claims he can predict the weather, months and even years in advance, by looking at lunar cycles – a claim not supported by science.

There are a couple of things at work here. For one, Ken Ring’s own explanation for how he arrives at his predictions is plausible sounding enough for the layperson. He uses the same scientific terms as the scientists do. This is often enough for most people to buy into. Combine with a liberal sprinkling of confirmation bias and voila, a pseudoscience is born.

After the initial preamble, Lewis continues:

Ring’s methods of using the moon and tides to forecast well beyond the timelines of MetService or NIWA have raised eyebrows in the scientific community for years, but the Kiwi weather watcher has been largely on the mark in New Zealand and overseas.

The next few paragraphs are citations of positive hits attributed to Ken Ring in order to support his claims.

The question then must be asked: why are these positive reports uncompelling to the scientific community? The answer is quite simple – what Lewis is putting forward as evidence for Ken Ring’s claims is purely observational selection bias. This is where one starts with an idea and finds supporting evidence for it while simultaneously disregarding contrary evidence.

Observational selection bias is very much a default setting of the brain and is one of the main reasons why human beings are not purely rational, objective recorders of reality. This is also why we test ideas scientifically. It is a common ploy by believers and proponents of all forms of pseudoscience to cherry pick data that supports their beliefs. However, it is only when the missing data is employed — the negative results — that we can get a complete picture of the validity of claims.

Tests smoke out bad ideas

In the case of Ken Ring’s “weather by the moon” ideas, the negative evidence is shows that his claims are not valid. In the case of a news article, negative results aren’t nearly as sexy or newsworthy and so are disregarded.

A similar article to Rebecca Lewis could be written about psychics and the conclusion could be drawn that the particular psychic in question was legitimate. All you would need to do is cite times when the psychic made hits and disregard the bulk of negatives that provide evidence that the psychic is not in fact legitimate.

A simple controlled scientific test would be the solution.

  1. Take people at random and ask them to predict the weather 12 months in advance. This would be pure guesswork on the part of the participants.
  2. You could then compare their hits and misses with those of Ken Ring’s moon predictions.
  3. If there were something to Ken’s claims, we would see a sizeable statistical difference from the control group.

I predict that we wouldn’t see that but as a test of this kind is yet to be done, Ken’s own claim, that anecdotal reports show his predictions to have an 85% success rate, are premature and misleading. Anecdotal reports are not strong form of evidence and are subject to the very biases that go into confirming beliefs, not verifiable facts. A scientific test would filter these biases out.

“Despite a huge following, Weather Watch analyst Phil Duncan is sceptical about Ring’s theories, saying it is too difficult to predict weather more than a month out.”

Ah… the argument from popularity. By this comment, Rebecca Lewis seems to be implying that skepticism of Ring’s ideas isn’t justified because he has a lot of followers. So do many cults, homeopathy, acupuncture and HIV deniers.

The argument from popularity is not a very compelling one at all when you look at how many popular beliefs people hang on to despite overwhelming evidence to the contrary. The amount of people who believe something to be true is completely irrelevant. The scientific method is indifferent to what people think (just think, if science were dependent on opinion we wouldn’t call it science).

I suspect that most people who believe in Ken Ring’s “weather by the moon” material aren’t aware that their beliefs are underpinned purely by confirmation bias.

A related issue with the story is why the writer finds Ken Ring’s work compelling in the first place, given the scientific consensus that his ideas reside under the category of quackery. The fact neither NIWA nor the MetService endorse his “science” and use it to make their weather and climate forecasts is a revealing fact. But Lewis disregards this as strong reason for skepticism of Ken Ring’s claims and therefore ends up grandstanding for an unscientific notion.

Lesson? Be Skeptical of Simple Answers

The idea that something as complex as the weather can be predicted by purely looking at lunar cycles is overly simplistic and therefore invalid. The implausibility of predicting weather by this method becomes all the more pronounced when attempting to foretell what the weather will be months and even years out simply by the moon.

If the moon were a good indicator of weather then we could expect there to be some regularity in weather phenomena year-on-year. To an extent, we do observe this in the changing of the seasons, but this is climate, not the weather and these observations are related to the axial tilt relative to the sun.

What we do observe when looking at meteorological data is this – the weather is random and subject to many influences that vary from year to year, independent of lunar cycles. We also find that variability in weather events occurs every year and can’t accurately be predicted until the necessary forces shaping that weather is in the melting pot (sometimes this can only be seen days or even hours ahead out). This is because weather is a chaotic system and such systems are in constant flux that we can never know ahead of time what is actually going to happen.

If the hypothesis that the moon is an accurate predictor of weather, we should expect weather to correlate with lunar cycles. The fact we do not see such a correlation is falsification of the hypothesis put forward by Ken Ring.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s