Barry Brill rips into NIWA’s showcase paper on the Hokitika weather adjustments — the one intended to demonstrate that data adjustment is science rather than art. – Richard Treadgold
In February 2010, NIWA published a showcase paper titled Creating a Composite Temperature Record for Hokitika, offering details of adjustments it made to the NZ Climate Database for the Hokitika Airport weather station — one of the “Seven-station Series” (7SS) which makes up the official New Zealand temperature record.
The NZ MetService measurements for Hokitika cover the 20th century and display no significant linear trend up or down. The temperature recorded here in 1900 was 11.8°C, while 2008 was 11.93°C, and the 30-year average from 1971–2000 was 11.74°C.
The NIWA-adjusted version, on the other hand, shows a linear warming trend of 1.3°C, largely brought about by downward adjustments in the Hokitika temperatures in the first half of the century. The justification for those adjustments has been cited repeatedly as being an Appendix to a university thesis submitted in 1981 (see “The Salinger Thesis”).
The Hokitika details were made public as a worked example of the adjustments that NIWA has made to all seven weather stations in the 7SS, in consequence of the Salinger thesis. Accordingly, the credibility of the entire project stands or falls on the strength of reasoning advanced for the Hokitika alterations.
But that reasoning is not sound.
Station locations
The NIWA publication, authored by Dr Brett Mullan, points out that there were two site changes — in 1943 and 1964 respectively — which raised the possibility that adjustments might be required to ensure homogeneity.
Mullan states that, in both cases, the older and newer sites were operated in parallel for lengthy periods, enabling accurate measurement of their relative ‘standard’ temperatures. To render the earlier sites comparable with the present Airport site, temperatures need to be adjusted downwards by a net 0.4°C during 1900–43 (site 1), and upwards by 0.3°C during 1944–64 (site 2).
Salinger’s methodology is entirely different. He did not rely on overlapping readings, but compared both Hokitika sites with Westport and Milford Sound.
He notes that the Airport station (site 3) was initially unsuccessful and needed to be shifted by 180 metres in October 1967, whilst the site 2 enclosure was overgrown with “very long grass” by January 1965, and observations in 1964 “became poor”.
Salinger was unaware of any overlap in 1943-5 and none is shown in the NZ Climate Database. Mullan explains that this is an error, which will be corrected in future.
Measurement problems
NIWA has introduced a further adjustment, which reduces the temperatures in 1900–12 by 1.3°C, and this accounts for the major part of the new warming trend.
[Note: Although the figure -1.3°C appears in Mullan’s Schedule of Adjustments, his co-dated paper uses a figure of -1.1°C. Mullan also refers to the 1945 adjustment as -0.3°C in the Schedule, but -0.4°C in the paper.]
The justification for assuming a discontinuity in 1912 is a “Note” the Metservice appended to the record in May 1930, which appears as an Appendix to Dr Mullan’s paper. The Note records a series of unhappy circumstances in respect to the early history of site 1 at Hokitika:
1. The timing of observations was changed in 1907.
2. A new screen and thermometers were provided in 1912. The enclosure was too small, the maximum bulb was apparently reading too high, and wind, cloud and humidity readings were dubious.
3. In 1918–20 there were suspected errors in the wet-bulb thermometers.
4. In 1926, the 63 sq ft enclosure was seen to be too small and enlarged to 2000 sq ft.
5. In 1928, ordinary wet and dry-bulb thermometers were substituted for the maximum and minimum wet bulbs.
The most obvious reaction to this jumble would be a declaration that all the pre-1930 readings were too inconsistent to be usable. However, if various pressures required NIWA to make the most of whatever was available, they could make a case for retaining the MetService data, subject to a prominent caveat (on the grounds that the note did not recommend a total discard).
NIWA followed a third course. They accepted the 1912–30 records without demur and sans any caveat or footnote. Before 1912, they discarded the MetService records and bravely set out to calculate the temperatures which ought to have occurred at Hokitika Town during the affected period. This was no mean task — a century later — especially as there were no pre-1930 temperature data recorded at any site near Hokitika, or anywhere else on the West Coast of the South Island.
The methodology was straightforward enough:
I. Assume that Hokitika Town would have a constant relativity with selected non-neighbouring stations both before and after 1912.
II. Ascertain the relativity between pre-1912 and post-1912 in the comparator stations, and apply that figure to derive pre-1912 temperatures in Hokitika Town.
The heroic assumption
To those familiar with New Zealand topography, it is counter-intuitive [Barry means incredible – ed.] to expect the highly unpredictable West Coast weather to remain in lock-step with weather variability anywhere else on Earth.
In discussing Hokitika, the Salinger thesis expressly notes: “It is difficult to compare a west coast station with east coast locations because they are in different response areas.”
Further, if changes in long-term weather in one region reliably mirror the changes in several other regions, then most of those regions could safely be omitted from any nationwide climate change series. But no climate studies have ever suggested that New Zealand is regionally coherent.
Whilst oscillations such as El Nino might broadly drive warmer or cooler years for the entire country, we are concerned here with a detailed regional series measured in tenths (sometimes hundredths) of a degree. Such exacting levels are wholly incompatible with a sweeping generalisation that any region is more-or-less the same as any other.
The scientific literature supports the notion of inter-station comparisons when those stations are near neighbours — but not otherwise (see below).
Comparable stations?
The 1981 thesis favours inter-comparisons between South Island stations. Dr Mullan notes that Salinger compared Hokitika with Christchurch, Lincoln, Dunedin and Nelson, but he (Mullan) was unable to replicate the calculations. There are serious problems with the data from those four stations, and nobody knows whether or how the thesis sought to tackle those issues (see “The Salinger Thesis” above).
Mullan argues instead that Hokitika’s temperature variations are well correlated with Auckland and all other western parts of the country, as well as Southland.
But how well? The figure he offers is a correlation of 0.7, which indicates that Hokitika’s variations diverge from those of Auckland up to one-third of the time.
Does this mean that NIWA has tried to cure a 10% error by resorting to a methodology with a 30% error rate?
NIWA’s audacious assumption could only be sustained if it showed that the temperature anomalies of two or more specified stations had a correlation of 0.95+ with those of Hokitika Town over a period wholly comparable with 1900–12. That has not been done, and almost certainly cannot be done.
50ft stacks of thermometers
But this comparator issue should not even be considered on a region-by-region basis. The comparability (if any) needs to be established at station-to-station level.
Mullan’s Figure 4 seems intended to show that the Hokitika grid-point (not the station location) has some similarity to the Auckland grid-point. This is meaningless. If Hokitika’s entire grid-point were homogeneous, there would be no need for adjustments when stations moved a few hundred metres within the town.
Weather stations do not measure changes in the climates of towns or regions. As climate guru Jim Hansen points out at The Elusive Absolute Surface Air Temperature, this would require “many 50 ft stacks of thermometers distributed evenly over the whole region, an obvious practical impossibility.”
Albert Park, Auckland
It seems clear that Mullan chose Albert Park in Auckland as his principal comparator simply because it is the only station in the Seven-station Series with (apparent) uninterrupted data during 1900–12.
But Albert Park did not commence until September 1909. Until that time, the Auckland measurements were taken from a site adjacent to the Auckland Museum — more distant from Albert Park than is Hokitika Airport from Hokitika Town.
In his peer-reviewed paper, published in the New Zealand Journal of Science in 1980, JWD Hessell defines a “major site change” for urban sites as being a horizontal move of more than 100m. And, as Dr Mullan says (page 1), “the data from different sites should not simply be appended without adjustment, since significant biases can be introduced when measurement sites are moved.”
No adjustments have been made to the pre-1910 Museum site temperatures to render them compatible with Albert Park.
The Mullan paper suggests that valid comparisons can be made between Hokitika Town and Albert Park throughout 1900–11. This is clearly incorrect, as the maximum comparison period is less than two years.
There is also a major problem with Albert Park’s post-1912 temperature records. According to Hessell (above) the station’s “continual upward trend of mean temperatures is statistically unacceptable” — having an acceptance probability of only 0.5%. The paper found that “the three considerations of sheltering, screen change and urbanisation all tended to increase reported maximum daily temperatures.”
Adjustments were subtracted when they should have been added. See Auckland warming is deception.
Albert Park was clearly a cot-case. If its numerous problems have been used to infect the Hokitika 1900–12 records, then Hokitika’s acceptance probability is likely to be even less than 0.5%.
Christchurch Gardens
Dr Mullan’s other comparator choice is Christchurch Gardens. He averages 13 years after 1912, and compares this with the inadequate four years of data available pre-1912.
In footnote 6 he concedes that “the year to year variations in Christchurch temperatures do not always match those at Hokitika … Christchurch is relatively warmer in years with stronger westerlies… We would expect all these correlations to be substantially weaker on the monthly timescale.”
Hessell drew attention to the shelter and urbanisation problems at this site, drawing a parallel with Albert Park. Salinger observed that the station’s records required “allowance to be made for inhomogeneity of urbanisation.”
Putting it all together
The composite adjusted Hokitika record now shows a linear trend for 1900–2008 of +1.3°C.
The shape of the curve, which appears in Figure 5, suggests that the first decade is out of step and is about 1.0°C lower than the following 100 years. The figures in the spreadsheet bear this out. NIWA’s 1.3°C reduction of the pre-1912 temperatures seems plainly wrong, and the curve would be much more plausible without it.
Dr Mullan thinks it reasonable to ask how the adjusted Hokitika series compares with records elsewhere in New Zealand, and selects two stations from NIWA’s 11-Station Series (Tauranga and Hamilton) to provide a test.
Although no reasons are offered for the selection of these two stations, their most glaring joint characteristic is the paucity of data for the sensitive pre-1920 period. Hamilton has a brief burst around 1907 (which is notably higher than the Hokitika curve, even after manipulation), whilst Tauranga has no data at all pre-1913.
Importantly, Dr Salinger’s thesis noted that, as a result of three site changes in 10 years, the Hamilton record “is of dubious value before October 1939.” He also warned that the record of the pre-1941 Tauranga site in Judea was “unreliable.”
Readers are left to wonder why Dr Mullan would select these two unsuitable records for his comparators — apart from his apparent belief that they confirmed his hypothesis.
Dr Mullan claims that the temporal trends are similar as between the three stations. But the Hon Dr Mapp told Parliament in answer to Question 1713 (2010) that a temporal trend could not be calculated for any site “which has years of missing raw data.”
These two unsuitable comparators throw even further doubt on NIWA’s 1900–12 adjustments.
Neighbours or strangers?
No scientific authority is cited for the “choose-a-comparator” methodology used in the Mullan paper and the Salinger thesis. This is because no authority exists.
The Mullan paper cites two authorities. One is a 1998 paper by Peterson et al. (co-authored by Dr Salinger), which traverses various methods used around the world. It refers to:
• a New Zealand example, described in Rhoades & Salinger (1995), which uses CUSUM plots in comparing nine neighbouring Canterbury stations.
• a 1995 Salinger paper which provides an Oceania example, where stations are too isolated for ‘neighbour’ comparisons.
• an Australian paper (Torok) where “a simple average of surrounding stations was found to be inadequate” and “stations with climates dissimilar to the candidate were not used.”
Although numerous alternative methods are canvassed in the paper by Peterson et al., none of them contemplate resort to “non-neighbour” stations. The New Zealand section (para 4.1.5) expressly notes: “For stations with several neighbours, the decision to adjust can be taken with some confidence. For isolated stations this is not the case.”
The Rhoades & Salinger approach is classified in Peterson et al as a subjective methodology, and it is rejected by Torok & Mitchell (1996) for that reason. But even this paper seeks to lay down some rules: “For temperature data, a site-change effect can be estimated by a difference between the target station and the weighted mean of neighbouring stations, comparing equal periods before and after the site change”.
The text also emphasises the need to “select only those neighbouring stations that have no site changes over the period of comparison”.
Boissonade (2002, pg 86)1 says: “In practice, it is extremely rare to find neighbouring stations that have exactly the same regional and local climatological signals as the target station and, at the same time, do not contain any discontinuities.”
Methodology not known to science
In response to Parliamentary Question 2857 (2010), the Hon Dr Wayne Mapp cites a passage from Easterling et al. (1995): “The determination of homogeneity must be made by comparison of time series from the station of interest (candidate) to those from nearby, closely related reference stations.”
The literature prudently emphasises objective rules, and always employs the terms “neighbour” or “nearby”. There is no authority whatever for the “non-neighbour comparison” approach taken by either the Mullan paper or the Salinger thesis.
Determining century-old Hokitika Town temperatures by reference to pre-adjusted and data-poor sites in a hotchpot of distant cities — including Auckland, Dunedin, Christchurch, Nelson, Tauranga and Hamilton — is not a methodology known to science.
Independent Test
When there is dispute about 100-year-old measurements, they are best left alone — they cannot be re-measured. But sometimes the geologic record allows the history of glaciers to be re-constructed, providing hard physical evidence from which old temperatures can be deduced.
Anderson et al (2008) publishes the results of the University of Canterbury’s ice flow model, which simulates the general retreat of Franz Josef glacier during the 20th century. In comparing their results with the Hokitika climate record, the authors refer to the three “corrections” developed by Salinger in 1981 and state:
We find that if the corrections are adjusted in the early part of the record to –0.9°C from 1894 to 1911, and the correction from 1911 to 1945 removed, the coupled model simulates glacier length variations much more closely (Fig 4).
With the corrections adjusted, some features of the coupled model become obvious. The glacier model simulates the general pattern of 20th century advance and retreat very well…
So the glacier records suggest that the Hokitika adjustments exaggerate the warming trend. The 1912 downwards adjustment ought to have been less and the 1945 adjustment ought not to have been made at all.
Conclusions
1. The warming trend at Hokitika weather station was created solely by recent NIWA adjustments (downwards) to the pre-1950 MetService records.
2. NIWA published the Hokitika paper to justify the “Salinger thesis adjustments” made to all stations in the NZ Temperature Record. But its methodology differs from Salinger’s in every material respect — except the use of “non-neighbour” comparisons.
3. Canterbury University studies of the advance and retreat of Franz Josef Glacier strongly suggest that NIWA’s 1912 and 1945 Hokitika adjustments are wrong.
4. Although five pre-1930 instrument problems are noted in the MetService records, NIWA selected only 1912 for adjustment.
5. For the 1900–25 comparison with Hokitika Town, it used Albert Park in Auckland and the Botanical Gardens in Christchurch. Station comparisons require equal periods before and after the discontinuity. But Albert Park did not open until late 1909, and Christchurch Gardens opened in 1905. Both these stations were declared unreliable by Hessell, and the NIWA method infected Hokitika with their numerous flaws.
6. NIWA attempts to confirm its outcome by comparing the adjusted Hokitika with two other stations. Tauranga did not open until 1913, and the pre-1912 data from Hamilton suggest the 1912 Hokitika adjustment was wrong. Further, the pre-1940 records of both these stations have been classified as “unreliable” by Salinger.
7. There are no studies to indicate that the New Zealand climate is regionally coherent. Both Salinger and NIWA recognise that Christchurch is poorly correlated with the West Coast. But even if there is a level of correlation between regional grids, this says nothing about the comparability of towns, let alone individual stations.
8. Both the scientific authorities cited by NIWA emphasise that comparisons are relevant only to “neighbouring” stations, using terms such as “nearby” and “closely related.” They also proscribe use of any comparators which experienced site changes during the comparison period.
1Boissonnade, A.C., Heitkemper, L.J. and Whitehead, D (2002), “Weather data: Cleaning and Enhancement”, chapter 5 of “Climate Risk and the Weather Market”, Risk Books.
Visits: 101
A very good summary. Nicely done, Barry.
Fantastic fisking of Mullan. Thank you for all your extensive research and effort.
is not a methodology known to science
Beautiful understatement, Barry
I understood the whole purpose of the Hokitika paper NIWA published in February was to show how the 7SS adjustments in the Salinger thesis worked.
NIWA said many times that the raw data was available in the Climate Database, and the methodology was fully described in the Salinger thesis, so anybody could put those two together and replicate all the adjustments. Finally, they published the Hokitika paper to illustrate how easy it all was.
And then we find that NIWA found it too hard. For the large 1912 adjustment, Mullan’s paper wasn’t able to reproduce Salinger’s method of comparing with Dunedin, Lincoln, Christchurch and Nelson, and compared with Auckland instead.
So will NIWA now offer another example which DOES apply Salinger methodology?
NIWA have indeed repeatedly stated that the data and the methodology are “available” and have criticised the Coalition as troublemakers for asking for the adjustments instead of producing their own.
The CCG blog has constantly pointed out the various deficiencies in NIWA’s answers to us and in the Parliament, such as in Rodney Hide remains rightly honourable and many others.
In this guest post, Barry has gone further, by delving into the actual dataset and the adjustments newly made by Mullan and comparing them with published papers and with Salinger’s newly-available thesis. His work is outstanding and provides a direct denial of what David Wratt, James Renwick and their supporters have said, for what they said was the method was not the method.
It will be interesting to see if Hot Topic or Sciblogs make any response to his factual, well-reasoned demolition of NIWA’s latest work on the national temperature. I don’t expect NIWA will respond just now, but concentrate on the approaching legal suit.
Surely the glaciers retreat or advance due to snowfall at higher altitudes. Therefore if theres a period beit a century or 2 where NZ has fewer cold fronts that produce snowfall at say above 2000m then there is less feed for the ice so it retreats and vise versa. Sounds like the circum antarctic current and PDO to me over the decades.
Hi Quentin,
I believe that’s partly true, it seems to be a balancing act between precipitation at the centre and melting/ablation at the edges. Dr Cliff Collier (Univ. of WA) wrote a nice summary a few years back, not sure if you’ve seen it before.
-Bob D
TVNZ CLOSEUP last night.
I see they (TVNZ) are firmly pushing the AGW agenda again. Pity the populace don’t seem to rate climate as very important by their own survey! maybe people are getting it now.