Jared Rennie

Maintenance and Streamlining of the Global Historical Climatology Network – Monthly (GHCN-M) Dataset

J Jared Rennie (NOAA Collaborators: Jay Lawrimore, Byron Gleason, Claude Williams, David Wuertz, Matt Menne, Russ Vose)

Background

Since the early 1990s the Global Historical Climatology Network-Monthly (GHCN-M) dataset has been an internationally recognized source of data for the study of observed variability and change in land surface temperature. It provides monthly mean temperature data for 7280 stations from 226 countries and territories, ongoing monthly updates of more than 2000 stations to support monitoring of current and evolving climate conditions, and homogeneity adjustments to remove non-climatic influences that can bias the observed temperature record. Version 3, which marks the first major revision to this dataset in over ten years, will become operational on May 1^st, 2011. This version introduces a number of improvements and changes that include consolidating “duplicate” series, updating records from recent decades, and the use of new approaches to homogenization and quality assurance.

Accomplishments

Accomplishments noted thus far include the following:

• Became familiar with entire GHCN-M processing, including ingest, quality control, and homogeneity adjustments
• Introduced new data sources to increase the amount of data for 7280 stations. This includes World Weather Record data, Antarctic stations from the British Antarctic Survey, and European data from the Royal Netherlands Meteorological Institute.
• Created a troubleshooting tool to ensure daily processing is running correctly and to archive data for future use.
  

Figure 1 Number of Stations in GHCN-M version 2 (dashed line) and version 3 (solid line)

Planned Work

Future planned activities for this research effort are to:

• Incorporate maximum temperature, minimum temperature, and precipitation into GHCN-M processing
• Increase the amount of stations through new datasets, as well as incorporating data from the Global Historical Climatology Network – Daily (GHCN-D) dataset.

Publications

Lawrimore, J.H., M.J. Menne, B.E. Gleason, C.N. Williams, D.B. Wuertz, R.S. Vose, and J.J. Rennie, 2011: An Overview of the Global Historical Climatology Network Monthly Mean Temperature Dataset, Version 3. To be submitted to the Journal of Geophysical Research – Atmospheres

Maintenance and Streamlining of the Hourly Precipitation Data (HPD) Network Dataset

J Jared Rennie (NOAA Collaborators: Jay Lawrimore, Stuart Hinson, Ron Ray, Matt Menne)

Background

For decades NOAA’s National Climatic Data Center (NCDC) has collected, quality controlled, and archived data from the COOP Fischer & Porter (F&P) network of more than 2000 stations. This dataset is produced through ongoing quality control processing that includes extensive manual review and intervention by a trained meteorological technician. Not only does this require extensive resources in time and personnel, it results in delays of up to six months before a month of observations are fully quality controlled and available to the public. Using new methods of automated quality control, an experimental HPD dataset has been developed. Still in testing phase, this dataset contains the more than 2000 F&P stations along with thousands of additional stations (e.g. ASOS, USCRN, NWS Hydrometeorological Automated Data System) that report hourly precipitation. These data have been quality controlled using a set of checks including checks for spikes, global extremes, gaps, and climatological outliers. Efforts are ongoing to implement a complete suite of quality control procedures developed through empirical assessments of false positive and flag rates. Once completed these new quality control procedures will replace the current process of manual review and editing which is part of the DSI-3240 Hourly Precipitation Dataset.

Figure 2 Spatial coverage of US stations in the experimental HPD dataset, which include data from COOP F&P, ASOS, USCRN and NWS HADS

Accomplishments

Accomplishments noted for this research activity include:

• Became familiar with the entire HPD processing, including ingest, and quality control
• Ensured processing runs daily, and debugged if issues arise
• Began process to build a suite of QC procedures through advanced statistics
  

Planned Work

The planned effort for this research work is to:

• Continue building upon the QC procedures to make the data more robust
• Dataset to become operational in 2012

Publications

Rennie, J.J., A. Wilson, J.H. Lawrimore, M.J. Menne, and R. Ray, 2011: Implementing New Quality Control and Processing Systems for Hourly Precipitation Data. Currently submitted to the 19^th Conference of Applied Climatology Here in Asheville, NC

Assistance with Data Rescue for the Global Databank

J Jared Rennie (NOAA Collaborators: Jay Lawrimore)

Background

Currently there is an international effort to develop a single comprehensive surface temperature databank. This databank will contain actual meteorological observations taken globally at monthly, daily, and sub-daily resolutions. This databank will be version controlled and seek to ascertain data provenance, preferably enabling researchers to drill down all the way to the original data record. It will also have associated metadata, including changes in instrumentation and station moves.

Accomplishments

(Began Work on January 3^rd, 2011)
Accomplishments noted for this activity include:

• Gathered data in its native format (stage 1), and converted them into a common format (stage 2)
• Worked with Jay Lawrimore on determining a valid data format, as well as assigning the proper data provenance tracking flags

Figure 3 Proposed databank stages (source: www.surfacetemperatures.org)

Planned Work

The planned research activities are to:

• Continuing conversion of data from stage 1 to stage 2 as data becomes available
• Begin merging process to convert stage 2 data into a master database (stage 3)

Release code to FTP to ensure transparency