Climate Data Store Tutorial

Climate Data Store Tutorial#

The Copernicus Climate Data Store (CDS) is the data access portal of the Copernicus Climate Change Service (C3S) and offers access to data and applications about the Earth’s past, present and future climate. In this notebook, you will find an overview of the type of data that can be found in it and practical examples on how to access and retrieve this data.

logo

Learning objectives 🧠#

Understanding the purpose and functionality of the CDS.
Learning to access data interactively via the web interface: familiarizing with the process of browsing datasets, selecting data, and submitting download forms via the CDS web interface.
Learning to access data programmatically using the CDS API: acquiring knowledge on installing the CDS API, obtaining API keys, and making data retrieval requests.

Target audience 🎯#

Anyone interested in accessing climate data from the Copernicus Climate Data Store (CDS) either interactively through the CDS web interface or programmatically using the CDS API.

Prerequisites and assumed knowledge 🔍#

Basic Programming Skills: Familiarity with programming concepts, particularly in Python, as the tutorial involves using Python libraries like cdsapi for accessing data programmatically.
Familiarity with API Usage: Understanding of Application Programming Interfaces (APIs) and how to use them to interact with web services will be helpful, particularly for accessing data programmatically through the CDS API.

Difficulty

1/5

Run the tutorial

WEKEO

WEkEO serves as the official platform of the European Centre for Medium-Range Weather Forecasts (ECMWF), offering access to an extensive array of climate data and tools for analysis and visualization. It provides a robust environment for conducting in-depth analysis and exploration of climate-related datasets. To learn more about WEkEO and its offerings, visit their website.

Run this notebook on WEKEO

Possible Cloud Services

While Kaggle, Binder, and Colab are popular options for running notebooks in the cloud, it’s essential to note that these are just a few among many available choices. Each platform has its unique features and capabilities, catering to diverse user needs and preferences.

Kaggle	Binder	Colab

Outline#

Data retrieval
1. Access data interactively via the CDS web interface
2. Access data programmatically with the CDS API
3. Further resources

1. Data retrieval#

All Copernicus Climate Data Store (CDS) data is public and you can have free access to it. Before being able to dowload data, you must Self-register at the CDS registration page (if you do not have an account yet), log into the CDS portal and accept the Terms and Conditions.

After being logged in and having accepted the terms, there are two ways to access data from the CDS:

interactively via the CDS web interface, or
programmatically with the CDS API

1.A. Access data interactively via the CDS web interface#

The CDS web interface allows you to interactively browse, select and download datasets offered by the CDS. First, under Datasets, you can browse and select the dataset you are interested in. In a second step, you can then specify details of the data download form you wish to submit.

Filter and select a dataset#

As a first step, you can browse and filter the dataset you are interested in. The Datasets interface allows you either to select data based on different categories, e.g. Product type, Variable domain, Spatial / Temporal coverage, or through a free text search. The list of datasets in the main window allows you to select the dataset you are interested in.

logo

Once you select a dataset you will get redirected to a data description section, which provides you an overview of the chosen dataset as well as the option to specify the dataset you would like to download and to submit the "Download data" form.

Submit the Download data form#

NOTE:
Under the tab Your requests in the main menu, you can monitor the status of your data requests.

logo

1.B. Access data programmatically with the CDS API#

The Climate Data Store Application Program Interface (CDS API) is a Python library which allows you to access data from the CDS programmatically. The library is available for both Python versions, Python 2.7.x and Python 3, but we recommend to use the library under Python 3. In order to use the CDS API, follow the steps below:

Install the CDS API key#

After Loging in to the CDS portal, go to the api-how-to page
Copy the CDS API key displayed in the black terminal window in a file under $HOME/.cdsapirc

Note: You find your CDS API key displayed in the black terminal box under the section Install the CDS API key. If you do not see a URL or key appear in the black terminal box, please refresh your browser tab.

logo

The code below creates the file under your current working directory. Make sure to replace the ################ with your personal CDS API key.

%%writefile ./. cdsapirc

url: https://cds.climate.copernicus.eu/api/v2
key: ###################

UsageError: unrecognized arguments: cdsapirc

Alternative: Set CDS API credentials manually#

Alternatively, you can also define variables for url and key. These variables can then be set when you define the cdsapi.Client(url=URL, key=KEY) in your script (just follow the notebook, you’ll get there).

Please note: in the workflow notebooks, we will use this modality and set manually the CDS API key information for the data retrievals.

URL = 'https://cds.climate.copernicus.eu/api/v2'
KEY = '############################'

Install the CDS API client#

The next step is to install the CDS API client. You can do this with the package management system pip.

!pip install cdsapi

Use the CDS API client for data access#

Once the CDS API is installed, it can be used to request data from the Climate Data Store.

At the end of the "Download data" form addressed in the section 1.A, you get two options to select: Show API request and Submit Form. If you want to access data programatically, you must choose Show API request. By doing so, the API request appears, which you can then copy. Below, you can see the data retrievalrequest generated by the search made in section 1.A.

logo

To make your search, you must paste the API request into your code and make these two changes:

Add the url and key variables that you defined earlier to the cdsapi.Client(), so that it is read cdsapi.Client(url=URL, key=KEY); and
Define the name of the file you will download, by substituting ‘download.nc’ (in the last line of the API request) with the name of your choice.

You can also change your search now by adding/removing variables or changing other aspects of the data, as in the following example.

Example: ERA5-Land hourly data from 1950 to present#

CDS API name: reanalysis-era5-land

Variable: ['10m_u_component_of_wind', '10m_v_component_of_wind','2m_temperature']

Year: [1981 to 2020]

Month: 12

Day: 15

Time: 12:00

Area: [60, -10, 35, 30] # North, West, South, East

Format: netcdf

NOTE:
This request makes use of the keyword `area`, which enable you to retrieve only a geographical subset. The bounding box information are set as follows: `[N, W, S, E]`. When this keyword is set, the data is automatically projected to a grid from [-180, 180].

import cdsapi
c = cdsapi.Client(url=URL, ker=KEY)
c.retrieve(
    'reanalysis-era5-land',
    {
        'variable': [
            '10m_u_component_of_wind', '10m_v_component_of_wind', '2m_temperature',
        ],
        'year': [
            '1981', '1982', '1983',
            '1984', '1985', '1986',
            '1987', '1988', '1989',
            '1990', '1991', '1992',
            '1993', '1994', '1995',
            '1996', '1997', '1998',
            '1999', '2000', '2001',
            '2002', '2003', '2004',
            '2005', '2006', '2007',
            '2008', '2009', '2010',
            '2011', '2012', '2013',
            '2014', '2015', '2016',
            '2017', '2018', '2019',
            '2020',
        ],
        'month': '12',
        'day': '15',
        'time': '12:00',
        'format': 'netcdf',
        'area': [
            60, -10, 35,
            30,
        ],
    },
    'era5-land_eur_1981_2020.nc')

NOTE:
For data originating from ECMWF's Meteorological and Archival System (MARS), data is stored on a grid with longitudes from 0 to 360 degrees. It can be reprojected to a regular geographic latitude-longitude grid, by setting the keyword argument area and grid. Per default, data is retrieved in GRIB. If you wish to retrieve the data in netCDF, you have to specify it by using the keyword argument format.

1.C. Further resources#

Key Messages to Take Home 📌#

The climate data available through the Copernicus Climate Data Store (CDS) can be accessed interactively via the CDS web interface and programmatically with the CDS API.
Before being able do download data, you must Self-register at the CDS registration page (if you do not have an account yet), log into the CDS portal and accept the Terms and Conditions.
The Datasets interface allows you to interactively browse and select datasets offered by the CDS.
Credentials for the CDS API can be found in the api-how-to page under the section Install the CDS API key.

This project is licensed under APACHE License 2.0. | View on GitHub

Climate Data Store Tutorial

Contents

Climate Data Store Tutorial#

Learning objectives 🧠#

Target audience 🎯#

Prerequisites and assumed knowledge 🔍#

WEKEO

Possible Cloud Services

Outline#

1. Data retrieval#

1.A. Access data interactively via the CDS web interface#

Filter and select a dataset#

Submit the Download data form#

1.B. Access data programmatically with the CDS API#

Install the CDS API key#

Alternative: Set CDS API credentials manually#

Install the CDS API client#

Use the CDS API client for data access#

Example: ERA5-Land hourly data from 1950 to present#

1.C. Further resources#

Key Messages to Take Home 📌#