Access Gridded Archive Data#
There are many times when we might want to access past (archive) datasets to analyze and visualize meteorological information. There are a host of different gridded data that are available through various THREDDS servers that allow simple remote access to the requested data. Unfortunately, these data are not all located in the same place or use the same conventions for storing the data. Data availability is also highly contigent on the date/time. Models get updated, new ones come along, and old ones are retired. Additionally, some datasets only contain the best analysis time (e.g., F000) for a given model run (at least 00, 06, 12, and 18), whereas others contain all of the forecast hours. Therefore you’ll want to look carefully at the data avalability times and the full scope of the available data to help you choose the best data access method for the question(s) you are attempting to answer.
from datetime import datetime
import metpy.calc as mpcalc
import xarray as xr
RDA GFS Analysis Archive#
The NCAR Research Data Archive (RDA) is a great resource for gridded data that can be easily utilized in the declarative syntax within MetPy. Depending on the dates desired, there are different end points to point to gain access to the desired data. This archive contains all of the forecast hours in addition to the analysis (e.g., F000) times.
GFS Data from 1 January 2009 - near current
GFS One Degree Archive - Older Data#
18 UTC 30 July 1999 through 06 UTC 6 December 7
# Data available July 30, 1999 at 18 UTC through December 6, 2007 at 06 UTC
ds = xr.open_dataset('https://thredds.rda.ucar.edu/thredds/dodsC/aggregations/g/ds083.2/1/TP')
print(ds.Geopotential_height_isobaric)
<xarray.DataArray 'Geopotential_height_isobaric' (time9: 12074, isobaric1: 26,
lat: 181, lon: 360)>
[20455287840 values with dtype=float32]
Coordinates:
* lat (lat) float32 90.0 89.0 88.0 87.0 ... -87.0 -88.0 -89.0 -90.0
* lon (lon) float32 0.0 1.0 2.0 3.0 4.0 ... 356.0 357.0 358.0 359.0
* time9 (time9) datetime64[ns] 1999-07-30T18:00:00 ... 2007-12-06T06:0...
reftime9 (time9) datetime64[ns] ...
* isobaric1 (isobaric1) float32 10.0 20.0 30.0 50.0 ... 950.0 975.0 1e+03
Attributes: (12/14)
long_name: Geopotential height @ Isobaric surface
units: gpm
description: Geopotential height
grid_mapping: LatLon_Projection
Grib_Variable_Id: VAR_7-0-2-7_L100
Grib1_Center: 7
... ...
Grib1_Parameter: 7
Grib1_Parameter_Name: HGT
Grib1_Level_Type: 100
Grib1_Level_Desc: Isobaric surface
Grib1_Interval_Type: -1
Grib1_Interval_Name: Unknown Time Range Indicator -1
GFS One Degree Archive - to current#
12 UTC 6 December 2007 to 18 UTC 13 March 2023
# Data Available December 6, 2007 at 06 UTC through March 31, 2023 at 18 UTC
ds = xr.open_dataset('https://thredds.rda.ucar.edu/thredds/dodsC/aggregations/g/ds083.2/2/TP')
print(ds.Geopotential_height_isobaric)
<xarray.DataArray 'Geopotential_height_isobaric' (time8: 22306, isobaric1: 41,
lat: 181, lon: 360)>
[59591817360 values with dtype=float32]
Coordinates:
* lat (lat) float32 90.0 89.0 88.0 87.0 ... -87.0 -88.0 -89.0 -90.0
* lon (lon) float32 0.0 1.0 2.0 3.0 4.0 ... 356.0 357.0 358.0 359.0
* time8 (time8) datetime64[ns] 2007-12-06T12:00:00 ... 2023-03-13T18:0...
reftime8 (time8) datetime64[ns] ...
* isobaric1 (isobaric1) float32 1.0 2.0 4.0 7.0 ... 9.5e+04 9.75e+04 1e+05
Attributes: (12/13)
long_name: Geopotential height @ Isobaric surface
units: gpm
abbreviation: HGT
grid_mapping: LatLon_Projection
Grib_Variable_Id: VAR_7-0--1-5_L100
Grib2_Parameter: [0 3 5]
... ...
Grib2_Parameter_Category: Mass
Grib2_Parameter_Name: Geopotential height
Grib2_Level_Type: 100
Grib2_Level_Desc: Isobaric surface
Grib2_Generating_Process_Type: Forecast
Grib2_Statistical_Process_Type: UnknownStatType--1
GFS 0.25 Degree Archive - to current#
12 UTC 6 December 2007 to present
date = datetime(2019, 3, 12, 0)
# Current data from January 15, 2015 at 00 UTC through present
# Direct file access
ds = xr.open_dataset('https://thredds.rda.ucar.edu/thredds/dodsC/files/g/ds084.1/'
f'{date:%Y}/{date:%Y%m%d}/gfs.0p25.{date:%Y%m%d%H}.f000.grib2')
print(ds.Geopotential_height_isobaric)
<xarray.DataArray 'Geopotential_height_isobaric' (time: 1, isobaric3: 31,
lat: 721, lon: 1440)>
[32185440 values with dtype=float32]
Coordinates:
* lat (lat) float32 90.0 89.75 89.5 89.25 ... -89.25 -89.5 -89.75 -90.0
* lon (lon) float32 0.0 0.25 0.5 0.75 1.0 ... 359.0 359.2 359.5 359.8
reftime datetime64[ns] ...
* time (time) datetime64[ns] 2019-03-12
* isobaric3 (isobaric3) float32 100.0 200.0 300.0 ... 9.5e+04 9.75e+04 1e+05
Attributes: (12/13)
long_name: Geopotential height @ Isobaric surface
units: gpm
abbreviation: HGT
grid_mapping: LatLon_Projection
Grib_Variable_Id: VAR_0-3-5_L100
Grib2_Parameter: [0 3 5]
... ...
Grib2_Parameter_Category: Mass
Grib2_Parameter_Name: Geopotential height
Grib2_Level_Type: 100
Grib2_Level_Desc: Isobaric surface
Grib2_Generating_Process_Type: Forecast
Grib2_Statistical_Process_Type: UnknownStatType--1
GFS 0.5 Degree Analysis Data#
Another set of archive gridded data are those available from NOAA NCEI. This data source has been more tempermental and is not always available. These files only contain the final analysis (e.g., F000) for each of the model runs (00, 06, 12, 18).
GFS Half Degree Data from December 2007 - present
date = datetime(2018, 12, 25, 0)
# GFS 0.5 degree Analysis back to 00 UTC 1 January 2007 to three years ago
ds = xr.open_dataset('https://www.ncei.noaa.gov/thredds/dodsC/model-gfs-g4-anl-files-old/'
f'{date:%Y%m/%Y%m%d}/gfsanl_4_{date:%Y%m%d_%H}00_000.grb2')
# GFS 0.5 degree Analysis for last three years to present (~ 3 days ago)
#ds = xr.open_dataset('https://www.ncei.noaa.gov/thredds/dodsC/model-gfs-g4-anl-files/'
# f'{date:%Y%m/%Y%m%d}/gfs_4_{date:%Y%m%d_%H}00_000.grb2')
print(ds.Geopotential_height_isobaric)
<xarray.DataArray 'Geopotential_height_isobaric' (time: 1, isobaric: 31,
lat: 361, lon: 720)>
[8057520 values with dtype=float32]
Coordinates:
* lat (lat) float32 90.0 89.5 89.0 88.5 88.0 ... -88.5 -89.0 -89.5 -90.0
* lon (lon) float32 0.0 0.5 1.0 1.5 2.0 ... 358.0 358.5 359.0 359.5
reftime datetime64[ns] ...
* time (time) datetime64[ns] 2018-12-25
* isobaric (isobaric) float32 100.0 200.0 300.0 ... 9.5e+04 9.75e+04 1e+05
Attributes:
long_name: Geopotential height @ Isobaric surface
units: gpm
abbreviation: HGT
grid_mapping: LatLon_Projection
Grib_Variable_Id: VAR_0-3-5_L100
Grib2_Parameter: [0 3 5]
Grib2_Parameter_Discipline: Meteorological products
Grib2_Parameter_Category: Mass
Grib2_Parameter_Name: Geopotential height
Grib2_Level_Type: 100
Grib2_Level_Desc: Isobaric surface
Grib2_Generating_Process_Type: Forecast
GFS 1 Degree Data#
Another set of archive gridded data are those available from NOAA NCEI. This data source has been more tempermental and is not always available. These files only contain the final analysis (e.g., F000) for each of the model runs (00, 06, 12, 18).
GFS One Degree Data from March 2004 - nearly present
date = datetime(2004, 3, 2, 0)
# GFS 1-degree Analysis back to 00 UTC 2 March 2004 to April 2020
# Dates after April 5, 2017 use the suffix of .grb2 instead of .grb
ds = xr.open_dataset('https://www.ncei.noaa.gov/thredds/dodsC/model-gfs-g3-anl-files-old/'
f'{date:%Y%m/%Y%m%d}/gfsanl_3_{date:%Y%m%d_%H}00_000.grb')
# GFS 1 degree Analysis from May 2020 to five days prior to present (~ 3 days ago)
#ds = xr.open_dataset('https://www.ncei.noaa.gov/thredds/dodsC/model-gfs-g3-anl-files/'
# f'{date:%Y%m/%Y%m%d}/gfs_3_{date:%Y%m%d_%H}00_000.grb2')
print(ds.Geopotential_height_isobaric)
<xarray.DataArray 'Geopotential_height_isobaric' (time: 1, isobaric2: 26,
lat: 181, lon: 360)>
[1694160 values with dtype=float32]
Coordinates:
* lat (lat) float32 90.0 89.0 88.0 87.0 ... -87.0 -88.0 -89.0 -90.0
* lon (lon) float32 0.0 1.0 2.0 3.0 4.0 ... 356.0 357.0 358.0 359.0
reftime datetime64[ns] ...
* time (time) datetime64[ns] 2004-03-02
* isobaric2 (isobaric2) float32 10.0 20.0 30.0 50.0 ... 950.0 975.0 1e+03
Attributes:
long_name: Geopotential height @ Isobaric surface
units: gpm
description: Geopotential height
grid_mapping: LatLon_Projection
Grib_Variable_Id: VAR_7-0-2-7_L100
Grib1_Center: 7
Grib1_Subcenter: 0
Grib1_TableVersion: 2
Grib1_Parameter: 7
Grib1_Parameter_Name: HGT
Grib1_Level_Type: 100
Grib1_Level_Desc: Isobaric surface
NAM 12 km Analysis Data#
nother set of archive gridded data are those available from NOAA NCEI. This data source has been more tempermental and is not always available. These files only contain the final analysis (e.g., F000) for each of the model runs (00, 06, 12, 18).
NAM 12 km Data from March 2004 - nearly present
date = datetime(2008, 3, 2, 18)
# NAM 218 Grid Analysis back to 18 UTC 2 March 2004 to three years ago
# Dates after April 5, 2017 use the suffix of .grb2 instead of .grb
ds = xr.open_dataset('https://www.ncei.noaa.gov/thredds/dodsC/model-namanl-old/'
f'{date:%Y%m/%Y%m%d}/namanl_218_{date:%Y%m%d_%H}00_000.grb')
# NAM 218 Grid Analysis for the past three years to near present (~ 3 days ago)
#ds = xr.open_dataset('https://www.ncei.noaa.gov/thredds/dodsC/model-namanl-old/'
# f'{date:%Y%m/%Y%m%d}/namanl_218_{date:%Y%m%d_%H}00_000.grb2')
print(ds.Geopotential_height_isobaric)
<xarray.DataArray 'Geopotential_height_isobaric' (time: 1, isobaric: 39,
y: 428, x: 614)>
[10248888 values with dtype=float32]
Coordinates:
* x (x) float32 -4.224e+03 -4.211e+03 ... 3.237e+03 3.249e+03
* y (y) float32 -832.2 -820.0 -807.8 ... 4.349e+03 4.361e+03 4.373e+03
reftime datetime64[ns] ...
* time (time) datetime64[ns] 2008-03-02T18:00:00
* isobaric (isobaric) float32 50.0 75.0 100.0 125.0 ... 950.0 975.0 1e+03
Attributes:
long_name: Geopotential height @ Isobaric surface
units: gpm
description: Geopotential height
grid_mapping: LambertConformal_Projection
Grib_Variable_Id: VAR_7-0-2-7_L100
Grib1_Center: 7
Grib1_Subcenter: 0
Grib1_TableVersion: 2
Grib1_Parameter: 7
Grib1_Parameter_Name: HGT
Grib1_Level_Type: 100
Grib1_Level_Desc: Isobaric surface
North American Regional Reanalysis#
Another set of archive gridded data are those available from NOAA NCEI. This data source has been more tempermental and is not always available. This is a reanalysis dataset that was stored in this location from September 2014. The data are on the NAM grid.
NARR Data from January 1979 - September 2014
date = datetime(1991, 10, 15, 0)
# North American Regional Reanalysis Analysis back to 0 UTC 1 January 1979 to 30 September 2014
ds = xr.open_dataset('https://www.ncei.noaa.gov/thredds/dodsC/model-narr-a-files/'
f'{date:%Y%m/%Y%m%d}/narr-a_221_{date:%Y%m%d_%H}00_000.grb')
print(ds.Geopotential_height_isobaric)
<xarray.DataArray 'Geopotential_height_isobaric' (time2: 1, isobaric1: 29,
y: 277, x: 349)>
[2803517 values with dtype=float32]
Coordinates:
* x (x) float32 -5.629e+03 -5.597e+03 ... 5.635e+03 5.668e+03
* y (y) float32 -4.61e+03 -4.577e+03 ... 4.317e+03 4.35e+03
reftime datetime64[ns] ...
* time2 (time2) datetime64[ns] 1991-10-15
* isobaric1 (isobaric1) float32 100.0 125.0 150.0 175.0 ... 950.0 975.0 1e+03
Attributes:
long_name: Geopotential height @ Isobaric surface
units: gpm
description: Geopotential height
grid_mapping: LambertConformal_Projection
Grib_Variable_Id: VAR_7-15-131-7_L100
Grib1_Center: 7
Grib1_Subcenter: 15
Grib1_TableVersion: 131
Grib1_Parameter: 7
Grib1_Level_Type: 100
Grib1_Level_Desc: Isobaric surface