Visualise data with xcube viewer

How to visualise data with xcube viewer in your workspace¶

A DeepESDL example notebook¶

This notebook demonstrates how to use the features of the xcube JupyterLab integration. The notebook demonstrates three scenarios how xcube Viewer is utilized in JupyterLab. In particular, we open xcube Viewer for any xarray.Dataset instances

1. persisted in team s3 storage (saved datasets)
2. opened or otherwise created in this Notebook (in-memory datasets)
3. using a configuration file for customising stlyes.

To explore the xcube viewer functionalities, please checkout the documentation: https://xcube.readthedocs.io/en/latest/viewer.html#functionality

Please, also refer to the DeepESDL documentation and visit the platform's website for further information!

Brockmann Consult, 2024

---

This notebook runs with the python environment deepesdl-xcube-1.7.0, please checkout the documentation for help on changing the environment.

First, lets create a small cube, which we can visualise with xcube viewer. We will use ESA CCI data for this. Please head over to example notebook in xcube-datastores: Generate CCI data cubes to get more details about the xcube-cci data store :)

In [1]:

from xcube.core.store import new_data_store
import os

from xcube.core.store import new_data_store import os

In [2]:

store = new_data_store('cciodp')
store

store = new_data_store('cciodp') store

Out[2]:

<xcube_cci.dataaccess.CciOdpDataStore at 0x7f9af651fa90>

We request a dataset from the datasore:

In [3]:

dataset = store.open_data('esacci.SST.day.L4.SSTdepth.multi-sensor.multi-platform.OSTIA.2-1.sst', 
                          variable_names=['analysed_sst'],
                          time_range=['1981-09-01','1981-09-07'])

dataset

dataset = store.open_data('esacci.SST.day.L4.SSTdepth.multi-sensor.multi-platform.OSTIA.2-1.sst', variable_names=['analysed_sst'], time_range=['1981-09-01','1981-09-07']) dataset

Out[3]:

<xarray.Dataset> Size: 1GB
Dimensions:       (time: 7, lat: 3600, lon: 7200, bnds: 2)
Coordinates:
  * lat           (lat) float32 14kB -89.97 -89.93 -89.88 ... 89.88 89.93 89.97
    lat_bnds      (lat, bnds) float32 29kB dask.array<chunksize=(3600, 2), meta=np.ndarray>
  * lon           (lon) float32 29kB -180.0 -179.9 -179.9 ... 179.9 179.9 180.0
    lon_bnds      (lon, bnds) float32 58kB dask.array<chunksize=(7200, 2), meta=np.ndarray>
  * time          (time) datetime64[ns] 56B 1981-09-01T12:00:00 ... 1981-09-0...
    time_bnds     (time, bnds) datetime64[ns] 112B dask.array<chunksize=(7, 2), meta=np.ndarray>
Dimensions without coordinates: bnds
Data variables:
    analysed_sst  (time, lat, lon) float64 1GB dask.array<chunksize=(1, 1200, 2400), meta=np.ndarray>
Attributes:
    Conventions:             CF-1.7
    title:                   esacci.SST.day.L4.SSTdepth.multi-sensor.multi-pl...
    date_created:            2024-09-18T08:07:18.447033
    processing_level:        L4
    time_coverage_start:     1981-09-01T00:00:00
    time_coverage_end:       1981-09-08T00:00:00
    time_coverage_duration:  P7DT0H0M0S
    history:                 [{'program': 'xcube_cci.chunkstore.CciChunkStore...

xarray.Dataset

• Dimensions:
  • time: 7
  • lat: 3600
  • lon: 7200
  • bnds: 2
• Coordinates: (6)
  • lat
    (lat)
    float32
    -89.97 -89.93 ... 89.93 89.97

    standard_name :

    latitude

    units :

    degrees_north

    valid_min :

    -90.0

    valid_max :

    90.0

    axis :

    Y

    comment :

    Latitude geographical coordinates,WGS84 projection

    long_name :

    Latitude

    reference_datum :

    geographical coordinates, WGS84 projection

    bounds :

    lat_bnds

    orig_data_type :

    float32

    fill_value :

    nan

    size :

    3600

    shape :

    [3600]

    chunk_sizes :

    3600

    file_chunk_sizes :

    3600

    data_type :

    float32

    dimensions :

    ['lat']

    file_dimensions :

    ['lat']

    array([-89.975, -89.925, -89.875, ...,  89.875,  89.925,  89.975],
          dtype=float32)

  • lat_bnds
    (lat, bnds)
    float32
    dask.array<chunksize=(3600, 2), meta=np.ndarray>

    units :

    degrees_north

    long_name :

    Latitude cell boundaries

    valid_min :

    -90.0

    valid_max :

    90.0

    comment :

    Contains the northern and southern boundaries of the grid cells.

    reference_datum :

    geographical coordinates, WGS84 projection

    orig_data_type :

    float32

    fill_value :

    nan

    size :

    7200

    shape :

    [3600, 2]

    chunk_sizes :

    [3600, 2]

    file_chunk_sizes :

    [3600, 2]

    data_type :

    float32

    dimensions :

    ['lat', 'bnds']

    file_dimensions :

    ['lat', 'bnds']

    Array Chunk Bytes 28.12 kiB 28.12 kiB Shape (3600, 2) (3600, 2) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • lon
    (lon)
    float32
    -180.0 -179.9 ... 179.9 180.0

    standard_name :

    longitude

    units :

    degrees_east

    valid_min :

    -180.0

    valid_max :

    180.0

    axis :

    X

    comment :

    Longitude geographical coordinates,WGS84 projection

    long_name :

    Longitude

    reference_datum :

    geographical coordinates, WGS84 projection

    bounds :

    lon_bnds

    orig_data_type :

    float32

    fill_value :

    nan

    size :

    7200

    shape :

    [7200]

    chunk_sizes :

    7200

    file_chunk_sizes :

    7200

    data_type :

    float32

    dimensions :

    ['lon']

    file_dimensions :

    ['lon']

    array([-179.975, -179.925, -179.875, ...,  179.875,  179.925,  179.975],
          dtype=float32)

  • lon_bnds
    (lon, bnds)
    float32
    dask.array<chunksize=(7200, 2), meta=np.ndarray>

    valid_max :

    180.0

    comment :

    Contains the eastern and western boundaries of the grid cells.

    reference_datum :

    geographical coordinates, WGS84 projection

    valid_min :

    -180.0

    units :

    degrees_east

    long_name :

    Longitude cell boundaries

    orig_data_type :

    float32

    fill_value :

    nan

    size :

    14400

    shape :

    [7200, 2]

    chunk_sizes :

    [7200, 2]

    file_chunk_sizes :

    [7200, 2]

    data_type :

    float32

    dimensions :

    ['lon', 'bnds']

    file_dimensions :

    ['lon', 'bnds']

    Array Chunk Bytes 56.25 kiB 56.25 kiB Shape (7200, 2) (7200, 2) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    1981-09-01T12:00:00 ... 1981-09-...

    standard_name :

    time

    bounds :

    time_bnds

    array(['1981-09-01T12:00:00.000000000', '1981-09-02T12:00:00.000000000',
           '1981-09-03T12:00:00.000000000', '1981-09-04T12:00:00.000000000',
           '1981-09-05T12:00:00.000000000', '1981-09-06T12:00:00.000000000',
           '1981-09-07T12:00:00.000000000'], dtype='datetime64[ns]')

  • time_bnds
    (time, bnds)
    datetime64[ns]
    dask.array<chunksize=(7, 2), meta=np.ndarray>

    standard_name :

    time_bnds

    Array Chunk Bytes 112 B 112 B Shape (7, 2) (7, 2) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

• Data variables: (1)
  • analysed_sst
    (time, lat, lon)
    float64
    dask.array<chunksize=(1, 1200, 2400), meta=np.ndarray>

    long_name :

    analysed sea surface temperature

    standard_name :

    sea_water_temperature

    units :

    kelvin

    valid_min :

    -300

    valid_max :

    4500

    source :

    ATSR<1,2>-ESACCI-L3U-v2.0, AATSR-ESACCI-L3U-v2.0, AVHRR<07,09,11,12,14,15,16,17,18,19>_G-ESACCI-L3U-v2.0, AVHRRMTA_G-ESACCI-L3U-v2.0

    depth :

    20 cm

    orig_data_type :

    int16

    fill_value :

    -32768

    size :

    181440000

    shape :

    [7, 3600, 7200]

    chunk_sizes :

    [1, 1200, 2400]

    file_chunk_sizes :

    [1, 1200, 2400]

    data_type :

    int16

    dimensions :

    ['time', 'lat', 'lon']

    file_dimensions :

    ['time', 'lat', 'lon']

    Array Chunk Bytes 1.35 GiB 21.97 MiB Shape (7, 3600, 7200) (1, 1200, 2400) Dask graph 63 chunks in 2 graph layers Data type float64 numpy.ndarray

• Indexes: (3)
  • lat
    PandasIndex

    PandasIndex(Index([ -89.9749984741211, -89.92500305175781,            -89.875,
           -89.82499694824219,  -89.7750015258789,  -89.7249984741211,
           -89.67500305175781,            -89.625, -89.57499694824219,
            -89.5250015258789,
           ...
             89.5250015258789,  89.57499694824219,             89.625,
            89.67500305175781,   89.7249984741211,   89.7750015258789,
            89.82499694824219,             89.875,  89.92500305175781,
             89.9749984741211],
          dtype='float32', name='lat', length=3600))

  • lon
    PandasIndex

    PandasIndex(Index([-179.97500610351562,  -179.9250030517578,            -179.875,
            -179.8249969482422, -179.77499389648438, -179.72500610351562,
            -179.6750030517578,            -179.625,  -179.5749969482422,
           -179.52499389648438,
           ...
            179.52499389648438,   179.5749969482422,             179.625,
             179.6750030517578,  179.72500610351562,  179.77499389648438,
             179.8249969482422,             179.875,   179.9250030517578,
            179.97500610351562],
          dtype='float32', name='lon', length=7200))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1981-09-01 12:00:00', '1981-09-02 12:00:00',
                   '1981-09-03 12:00:00', '1981-09-04 12:00:00',
                   '1981-09-05 12:00:00', '1981-09-06 12:00:00',
                   '1981-09-07 12:00:00'],
                  dtype='datetime64[ns]', name='time', freq=None))

• Attributes: (8)

  Conventions :

  CF-1.7

  title :

  esacci.SST.day.L4.SSTdepth.multi-sensor.multi-platform.OSTIA.2-1.sst

  date_created :

  2024-09-18T08:07:18.447033

  processing_level :

  L4

  time_coverage_start :

  1981-09-01T00:00:00

  time_coverage_end :

  1981-09-08T00:00:00

  time_coverage_duration :

  P7DT0H0M0S

  history :

  [{'program': 'xcube_cci.chunkstore.CciChunkStore', 'cube_params': {'variable_names': ['analysed_sst'], 'time_range': ['1981-09-01T00:00:00', '1981-09-07T00:00:00']}}]

Next, save it to the team s3 storage:

To store the cube in your teams user space, please first retrieve the details from your environment variables as the following:

In [4]:

S3_USER_STORAGE_KEY = os.environ["S3_USER_STORAGE_KEY"]
S3_USER_STORAGE_SECRET = os.environ["S3_USER_STORAGE_SECRET"]
S3_USER_STORAGE_BUCKET = os.environ["S3_USER_STORAGE_BUCKET"]

S3_USER_STORAGE_KEY = os.environ["S3_USER_STORAGE_KEY"] S3_USER_STORAGE_SECRET = os.environ["S3_USER_STORAGE_SECRET"] S3_USER_STORAGE_BUCKET = os.environ["S3_USER_STORAGE_BUCKET"]

You need to instantiate a s3 datastore pointing to the team bucket:

In [5]:

team_store = new_data_store("s3", 
                       root=S3_USER_STORAGE_BUCKET, 
                       storage_options=dict(anon=False, 
                                            key=S3_USER_STORAGE_KEY, 
                                            secret=S3_USER_STORAGE_SECRET))

team_store = new_data_store("s3", root=S3_USER_STORAGE_BUCKET, storage_options=dict(anon=False, key=S3_USER_STORAGE_KEY, secret=S3_USER_STORAGE_SECRET))

If you have stored no data to your user space, the returned list will be empty:

In [6]:

list(team_store.get_data_ids())

list(team_store.get_data_ids())

Out[6]:

['SST.levels',
 'amazonas_v8.zarr',
 'amazonas_v9.zarr',
 'analysed_sst.zarr',
 'noise_trajectory.zarr']

The writing will take a few moments, as the data is global and will be persisted into the team s3 storage.

In [7]:

output_id = 'analysed_sst.zarr'

output_id = 'analysed_sst.zarr'

In [10]:

team_store.write_data(dataset, output_id)

team_store.write_data(dataset, output_id)

Out[10]:

'analysed_sst.zarr'

If you list the content of you datastore again, you will now see the newly written dataset in the list:

In [11]:

list(team_store.get_data_ids())

list(team_store.get_data_ids())

Out[11]:

['SST.levels',
 'amazonas_v8.zarr',
 'amazonas_v9.zarr',
 'analysed_sst.zarr',
 'noise_trajectory.zarr']

Once the cube is stored in our team s3 storage, we can use xcube viewer jupyterlab extention to visualise it.

In [12]:

from xcube.webapi.viewer import Viewer

from xcube.webapi.viewer import Viewer

We use the xcube datastore framework here to open the dataset, but it could also be opened by other means, e.g., xr.open_dataset(), provided it has variables with dimensions ["time", "y", "x"] or ["y", "x"].

In [13]:

dataset = team_store.open_data(output_id)

dataset = team_store.open_data(output_id)

In [14]:

dataset

dataset

Out[14]:

<xarray.Dataset> Size: 1GB
Dimensions:       (time: 7, lat: 3600, lon: 7200, bnds: 2)
Coordinates:
  * lat           (lat) float32 14kB -89.97 -89.93 -89.88 ... 89.88 89.93 89.97
    lat_bnds      (lat, bnds) float32 29kB dask.array<chunksize=(3600, 2), meta=np.ndarray>
  * lon           (lon) float32 29kB -180.0 -179.9 -179.9 ... 179.9 179.9 180.0
    lon_bnds      (lon, bnds) float32 58kB dask.array<chunksize=(7200, 2), meta=np.ndarray>
  * time          (time) datetime64[ns] 56B 1981-09-01T12:00:00 ... 1981-09-0...
    time_bnds     (time, bnds) datetime64[ns] 112B dask.array<chunksize=(7, 2), meta=np.ndarray>
Dimensions without coordinates: bnds
Data variables:
    analysed_sst  (time, lat, lon) float64 1GB dask.array<chunksize=(1, 1200, 2400), meta=np.ndarray>
Attributes:
    Conventions:             CF-1.7
    date_created:            2024-09-18T08:07:18.447033
    history:                 [{'cube_params': {'time_range': ['1981-09-01T00:...
    processing_level:        L4
    time_coverage_duration:  P7DT0H0M0S
    time_coverage_end:       1981-09-08T00:00:00
    time_coverage_start:     1981-09-01T00:00:00
    title:                   esacci.SST.day.L4.SSTdepth.multi-sensor.multi-pl...

xarray.Dataset

• Dimensions:
  • time: 7
  • lat: 3600
  • lon: 7200
  • bnds: 2
• Coordinates: (6)
  • lat
    (lat)
    float32
    -89.97 -89.93 ... 89.93 89.97

    axis :

    Y

    bounds :

    lat_bnds

    chunk_sizes :

    3600

    comment :

    Latitude geographical coordinates,WGS84 projection

    data_type :

    float32

    dimensions :

    ['lat']

    file_chunk_sizes :

    3600

    file_dimensions :

    ['lat']

    fill_value :

    nan

    long_name :

    Latitude

    orig_data_type :

    float32

    reference_datum :

    geographical coordinates, WGS84 projection

    shape :

    [3600]

    size :

    3600

    standard_name :

    latitude

    units :

    degrees_north

    valid_max :

    90.0

    valid_min :

    -90.0

    array([-89.975, -89.925, -89.875, ...,  89.875,  89.925,  89.975],
          dtype=float32)

  • lat_bnds
    (lat, bnds)
    float32
    dask.array<chunksize=(3600, 2), meta=np.ndarray>

    chunk_sizes :

    [3600, 2]

    comment :

    Contains the northern and southern boundaries of the grid cells.

    data_type :

    float32

    dimensions :

    ['lat', 'bnds']

    file_chunk_sizes :

    [3600, 2]

    file_dimensions :

    ['lat', 'bnds']

    fill_value :

    nan

    long_name :

    Latitude cell boundaries

    orig_data_type :

    float32

    reference_datum :

    geographical coordinates, WGS84 projection

    shape :

    [3600, 2]

    size :

    7200

    units :

    degrees_north

    valid_max :

    90.0

    valid_min :

    -90.0

    Array Chunk Bytes 28.12 kiB 28.12 kiB Shape (3600, 2) (3600, 2) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • lon
    (lon)
    float32
    -180.0 -179.9 ... 179.9 180.0

    axis :

    X

    bounds :

    lon_bnds

    chunk_sizes :

    7200

    comment :

    Longitude geographical coordinates,WGS84 projection

    data_type :

    float32

    dimensions :

    ['lon']

    file_chunk_sizes :

    7200

    file_dimensions :

    ['lon']

    fill_value :

    nan

    long_name :

    Longitude

    orig_data_type :

    float32

    reference_datum :

    geographical coordinates, WGS84 projection

    shape :

    [7200]

    size :

    7200

    standard_name :

    longitude

    units :

    degrees_east

    valid_max :

    180.0

    valid_min :

    -180.0

    array([-179.975, -179.925, -179.875, ...,  179.875,  179.925,  179.975],
          dtype=float32)

  • lon_bnds
    (lon, bnds)
    float32
    dask.array<chunksize=(7200, 2), meta=np.ndarray>

    chunk_sizes :

    [7200, 2]

    comment :

    Contains the eastern and western boundaries of the grid cells.

    data_type :

    float32

    dimensions :

    ['lon', 'bnds']

    file_chunk_sizes :

    [7200, 2]

    file_dimensions :

    ['lon', 'bnds']

    fill_value :

    nan

    long_name :

    Longitude cell boundaries

    orig_data_type :

    float32

    reference_datum :

    geographical coordinates, WGS84 projection

    shape :

    [7200, 2]

    size :

    14400

    units :

    degrees_east

    valid_max :

    180.0

    valid_min :

    -180.0

    Array Chunk Bytes 56.25 kiB 56.25 kiB Shape (7200, 2) (7200, 2) Dask graph 1 chunks in 2 graph layers Data type float32 numpy.ndarray

  • time
    (time)
    datetime64[ns]
    1981-09-01T12:00:00 ... 1981-09-...

    bounds :

    time_bnds

    standard_name :

    time

    array(['1981-09-01T12:00:00.000000000', '1981-09-02T12:00:00.000000000',
           '1981-09-03T12:00:00.000000000', '1981-09-04T12:00:00.000000000',
           '1981-09-05T12:00:00.000000000', '1981-09-06T12:00:00.000000000',
           '1981-09-07T12:00:00.000000000'], dtype='datetime64[ns]')

  • time_bnds
    (time, bnds)
    datetime64[ns]
    dask.array<chunksize=(7, 2), meta=np.ndarray>

    standard_name :

    time_bnds

    Array Chunk Bytes 112 B 112 B Shape (7, 2) (7, 2) Dask graph 1 chunks in 2 graph layers Data type datetime64[ns] numpy.ndarray

• Data variables: (1)
  • analysed_sst
    (time, lat, lon)
    float64
    dask.array<chunksize=(1, 1200, 2400), meta=np.ndarray>

    chunk_sizes :

    [1, 1200, 2400]

    data_type :

    int16

    depth :

    20 cm

    dimensions :

    ['time', 'lat', 'lon']

    file_chunk_sizes :

    [1, 1200, 2400]

    file_dimensions :

    ['time', 'lat', 'lon']

    fill_value :

    -32768

    long_name :

    analysed sea surface temperature

    orig_data_type :

    int16

    shape :

    [7, 3600, 7200]

    size :

    181440000

    source :

    ATSR<1,2>-ESACCI-L3U-v2.0, AATSR-ESACCI-L3U-v2.0, AVHRR<07,09,11,12,14,15,16,17,18,19>_G-ESACCI-L3U-v2.0, AVHRRMTA_G-ESACCI-L3U-v2.0

    standard_name :

    sea_water_temperature

    units :

    kelvin

    valid_max :

    4500

    valid_min :

    -300

    Array Chunk Bytes 1.35 GiB 21.97 MiB Shape (7, 3600, 7200) (1, 1200, 2400) Dask graph 63 chunks in 2 graph layers Data type float64 numpy.ndarray

• Indexes: (3)
  • lat
    PandasIndex

    PandasIndex(Index([ -89.9749984741211, -89.92500305175781,            -89.875,
           -89.82499694824219,  -89.7750015258789,  -89.7249984741211,
           -89.67500305175781,            -89.625, -89.57499694824219,
            -89.5250015258789,
           ...
             89.5250015258789,  89.57499694824219,             89.625,
            89.67500305175781,   89.7249984741211,   89.7750015258789,
            89.82499694824219,             89.875,  89.92500305175781,
             89.9749984741211],
          dtype='float32', name='lat', length=3600))

  • lon
    PandasIndex

    PandasIndex(Index([-179.97500610351562,  -179.9250030517578,            -179.875,
            -179.8249969482422, -179.77499389648438, -179.72500610351562,
            -179.6750030517578,            -179.625,  -179.5749969482422,
           -179.52499389648438,
           ...
            179.52499389648438,   179.5749969482422,             179.625,
             179.6750030517578,  179.72500610351562,  179.77499389648438,
             179.8249969482422,             179.875,   179.9250030517578,
            179.97500610351562],
          dtype='float32', name='lon', length=7200))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['1981-09-01 12:00:00', '1981-09-02 12:00:00',
                   '1981-09-03 12:00:00', '1981-09-04 12:00:00',
                   '1981-09-05 12:00:00', '1981-09-06 12:00:00',
                   '1981-09-07 12:00:00'],
                  dtype='datetime64[ns]', name='time', freq=None))

• Attributes: (8)

  Conventions :

  CF-1.7

  date_created :

  2024-09-18T08:07:18.447033

  history :

  [{'cube_params': {'time_range': ['1981-09-01T00:00:00', '1981-09-07T00:00:00'], 'variable_names': ['analysed_sst']}, 'program': 'xcube_cci.chunkstore.CciChunkStore'}]

  processing_level :

  L4

  time_coverage_duration :

  P7DT0H0M0S

  time_coverage_end :

  1981-09-08T00:00:00

  time_coverage_start :

  1981-09-01T00:00:00

  title :

  esacci.SST.day.L4.SSTdepth.multi-sensor.multi-platform.OSTIA.2-1.sst

---

Scenario 1: Open xcube Viewer for a dataset instances persisted from a certain source (saved datasets).

For the functionalities of xcube viewer please head over to the documentation: https://xcube.readthedocs.io/en/latest/viewer.html#functionality

In [15]:

viewer = Viewer()

viewer = Viewer()

WARNING:tornado.general:404 GET /viewer/config/config.json (127.0.0.1): xcube viewer has not been been configured
WARNING:tornado.general:404 GET /viewer/config/config.json (127.0.0.1): xcube viewer has not been been configured
WARNING:tornado.access:404 GET /viewer/config/config.json (127.0.0.1) 4.77ms
WARNING:tornado.access:404 GET /viewer/config/config.json (127.0.0.1) 4.77ms

In [16]:

viewer.add_dataset(dataset)

viewer.add_dataset(dataset)

Out[16]:

'11098dfc-54a0-4462-8de3-cc3abfd36d5f'

You can click on the viewer link to open xcube Viewer in a new browser tab:

In [17]:

viewer.info()

viewer.info()

Server: https://deep.earthsystemdatalab.net/user/alicebalfanz/proxy/8001
Viewer: https://deep.earthsystemdatalab.net/user/alicebalfanz/proxy/8001/viewer/?serverUrl=https://deep.earthsystemdatalab.net/user/alicebalfanz/proxy/8001

You can also open xcube Viewer inlined here:

In [18]:

viewer.show()

viewer.show()

Out[18]:

To stop the server and viewer:

In [19]:

viewer.stop_server()

viewer.stop_server()

---

Scenario 2: Open xcube Viewer for a dataset instances opened or otherwise created in this Notebook (in-memory datasets).

Below, let's fetch a dataset from CDS on the fly, without persisting it - for more details about xcube CDS datastore and how to get CDS credentials please checkout the example notebook in xcube-datastores GENERATE C3S CDS CUBES. Be aware of performance loss, so in case you plan to use a dataset a lot for analysis or visualisation, please persist it into the team s3 storage. The public cubes provided within DeepESDL are already persisted in S3, so you should not duplicate them in your team storage.

In [20]:

import os
os.environ['CDSAPI_URL'] = 'https://cds-beta.climate.copernicus.eu/api'
os.environ['CDSAPI_KEY'] = '[PERSONAL-ACCESS-TOKEN]'

import os os.environ['CDSAPI_URL'] = 'https://cds-beta.climate.copernicus.eu/api' os.environ['CDSAPI_KEY'] = '[PERSONAL-ACCESS-TOKEN]'

In [22]:

cds_store = new_data_store('cds')

cds_store = new_data_store('cds')

In [44]:

bbox=[-5, 45, 35, 65]

bbox=[-5, 45, 35, 65]

In [47]:

cds_dataset = cds_store.open_data('reanalysis-era5-single-levels-monthly-means:monthly_averaged_reanalysis', 
                          variable_names=['2m_temperature'], 
                          bbox=bbox, 
                          spatial_res=0.25, 
                          time_range=['2010-01-01', '2010-12-31'])
cds_dataset

cds_dataset = cds_store.open_data('reanalysis-era5-single-levels-monthly-means:monthly_averaged_reanalysis', variable_names=['2m_temperature'], bbox=bbox, spatial_res=0.25, time_range=['2010-01-01', '2010-12-31']) cds_dataset

xcube-cds version 0.9.3
2024-09-18 08:40:41,936 INFO Request ID is 6a55691a-15bb-4b7d-81dc-fedb6febd390
INFO:cads_api_client.processing:Request ID is 6a55691a-15bb-4b7d-81dc-fedb6febd390
2024-09-18 08:40:41,973 INFO status has been updated to accepted
INFO:cads_api_client.processing:status has been updated to accepted
2024-09-18 08:40:43,517 INFO status has been updated to running
INFO:cads_api_client.processing:status has been updated to running
2024-09-18 08:40:45,804 INFO status has been updated to successful
INFO:cads_api_client.processing:status has been updated to successful

9830c36a333f3eab454e3817a76610df.nc:   0%|          | 0.00/322k [00:00<?, ?B/s]

Out[47]:

<xarray.Dataset> Size: 617kB
Dimensions:  (lat: 80, lon: 160, time: 12)
Coordinates:
    number   int64 8B ...
  * lat      (lat) float64 640B 64.88 64.62 64.38 64.12 ... 45.62 45.38 45.12
  * lon      (lon) float64 1kB -4.875 -4.625 -4.375 -4.125 ... 34.38 34.62 34.88
    expver   (time) <U4 192B ...
  * time     (time) datetime64[ns] 96B 2010-01-01 2010-02-01 ... 2010-12-01
Data variables:
    t2m      (time, lat, lon) float32 614kB ...
Attributes:
    GRIB_centre:             ecmf
    GRIB_centreDescription:  European Centre for Medium-Range Weather Forecasts
    GRIB_subCentre:          0
    Conventions:             CF-1.7
    institution:             European Centre for Medium-Range Weather Forecasts
    history:                 2024-09-18T08:40 GRIB to CDM+CF via cfgrib-0.9.1...

xarray.Dataset

• Dimensions:
  • lat: 80
  • lon: 160
  • time: 12
• Coordinates: (5)
  • number
    ()
    int64
    ...

    long_name :

    ensemble member numerical id

    units :

    1

    standard_name :

    realization

    [1 values with dtype=int64]

  • lat
    (lat)
    float64
    64.88 64.62 64.38 ... 45.38 45.12

    units :

    degrees_north

    standard_name :

    latitude

    long_name :

    latitude

    stored_direction :

    decreasing

    array([64.875, 64.625, 64.375, 64.125, 63.875, 63.625, 63.375, 63.125, 62.875,
           62.625, 62.375, 62.125, 61.875, 61.625, 61.375, 61.125, 60.875, 60.625,
           60.375, 60.125, 59.875, 59.625, 59.375, 59.125, 58.875, 58.625, 58.375,
           58.125, 57.875, 57.625, 57.375, 57.125, 56.875, 56.625, 56.375, 56.125,
           55.875, 55.625, 55.375, 55.125, 54.875, 54.625, 54.375, 54.125, 53.875,
           53.625, 53.375, 53.125, 52.875, 52.625, 52.375, 52.125, 51.875, 51.625,
           51.375, 51.125, 50.875, 50.625, 50.375, 50.125, 49.875, 49.625, 49.375,
           49.125, 48.875, 48.625, 48.375, 48.125, 47.875, 47.625, 47.375, 47.125,
           46.875, 46.625, 46.375, 46.125, 45.875, 45.625, 45.375, 45.125])

  • lon
    (lon)
    float64
    -4.875 -4.625 ... 34.62 34.88

    units :

    degrees_east

    standard_name :

    longitude

    long_name :

    longitude

    array([-4.875, -4.625, -4.375, -4.125, -3.875, -3.625, -3.375, -3.125, -2.875,
           -2.625, -2.375, -2.125, -1.875, -1.625, -1.375, -1.125, -0.875, -0.625,
           -0.375, -0.125,  0.125,  0.375,  0.625,  0.875,  1.125,  1.375,  1.625,
            1.875,  2.125,  2.375,  2.625,  2.875,  3.125,  3.375,  3.625,  3.875,
            4.125,  4.375,  4.625,  4.875,  5.125,  5.375,  5.625,  5.875,  6.125,
            6.375,  6.625,  6.875,  7.125,  7.375,  7.625,  7.875,  8.125,  8.375,
            8.625,  8.875,  9.125,  9.375,  9.625,  9.875, 10.125, 10.375, 10.625,
           10.875, 11.125, 11.375, 11.625, 11.875, 12.125, 12.375, 12.625, 12.875,
           13.125, 13.375, 13.625, 13.875, 14.125, 14.375, 14.625, 14.875, 15.125,
           15.375, 15.625, 15.875, 16.125, 16.375, 16.625, 16.875, 17.125, 17.375,
           17.625, 17.875, 18.125, 18.375, 18.625, 18.875, 19.125, 19.375, 19.625,
           19.875, 20.125, 20.375, 20.625, 20.875, 21.125, 21.375, 21.625, 21.875,
           22.125, 22.375, 22.625, 22.875, 23.125, 23.375, 23.625, 23.875, 24.125,
           24.375, 24.625, 24.875, 25.125, 25.375, 25.625, 25.875, 26.125, 26.375,
           26.625, 26.875, 27.125, 27.375, 27.625, 27.875, 28.125, 28.375, 28.625,
           28.875, 29.125, 29.375, 29.625, 29.875, 30.125, 30.375, 30.625, 30.875,
           31.125, 31.375, 31.625, 31.875, 32.125, 32.375, 32.625, 32.875, 33.125,
           33.375, 33.625, 33.875, 34.125, 34.375, 34.625, 34.875])

  • expver
    (time)
    <U4
    ...

    [12 values with dtype=<U4]

  • time
    (time)
    datetime64[ns]
    2010-01-01 ... 2010-12-01

    standard_name :

    time

    array(['2010-01-01T00:00:00.000000000', '2010-02-01T00:00:00.000000000',
           '2010-03-01T00:00:00.000000000', '2010-04-01T00:00:00.000000000',
           '2010-05-01T00:00:00.000000000', '2010-06-01T00:00:00.000000000',
           '2010-07-01T00:00:00.000000000', '2010-08-01T00:00:00.000000000',
           '2010-09-01T00:00:00.000000000', '2010-10-01T00:00:00.000000000',
           '2010-11-01T00:00:00.000000000', '2010-12-01T00:00:00.000000000'],
          dtype='datetime64[ns]')

• Data variables: (1)
  • t2m
    (time, lat, lon)
    float32
    ...

    GRIB_paramId :

    167

    GRIB_dataType :

    an

    GRIB_numberOfPoints :

    12800

    GRIB_typeOfLevel :

    surface

    GRIB_stepUnits :

    1

    GRIB_stepType :

    avgua

    GRIB_gridType :

    regular_ll

    GRIB_uvRelativeToGrid :

    0

    GRIB_NV :

    0

    GRIB_Nx :

    160

    GRIB_Ny :

    80

    GRIB_cfName :

    unknown

    GRIB_cfVarName :

    t2m

    GRIB_gridDefinitionDescription :

    Latitude/Longitude Grid

    GRIB_iDirectionIncrementInDegrees :

    0.25

    GRIB_iScansNegatively :

    0

    GRIB_jDirectionIncrementInDegrees :

    0.25

    GRIB_jPointsAreConsecutive :

    0

    GRIB_jScansPositively :

    0

    GRIB_latitudeOfFirstGridPointInDegrees :

    64.875

    GRIB_latitudeOfLastGridPointInDegrees :

    45.125

    GRIB_longitudeOfFirstGridPointInDegrees :

    -4.875

    GRIB_longitudeOfLastGridPointInDegrees :

    34.875

    GRIB_missingValue :

    3.4028234663852886e+38

    GRIB_name :

    2 metre temperature

    GRIB_shortName :

    2t

    GRIB_totalNumber :

    0

    GRIB_units :

    K

    long_name :

    2 metre temperature

    units :

    K

    standard_name :

    unknown

    GRIB_surface :

    0.0

    [153600 values with dtype=float32]

• Indexes: (3)
  • lat
    PandasIndex

    PandasIndex(Index([64.875, 64.625, 64.375, 64.125, 63.875, 63.625, 63.375, 63.125, 62.875,
           62.625, 62.375, 62.125, 61.875, 61.625, 61.375, 61.125, 60.875, 60.625,
           60.375, 60.125, 59.875, 59.625, 59.375, 59.125, 58.875, 58.625, 58.375,
           58.125, 57.875, 57.625, 57.375, 57.125, 56.875, 56.625, 56.375, 56.125,
           55.875, 55.625, 55.375, 55.125, 54.875, 54.625, 54.375, 54.125, 53.875,
           53.625, 53.375, 53.125, 52.875, 52.625, 52.375, 52.125, 51.875, 51.625,
           51.375, 51.125, 50.875, 50.625, 50.375, 50.125, 49.875, 49.625, 49.375,
           49.125, 48.875, 48.625, 48.375, 48.125, 47.875, 47.625, 47.375, 47.125,
           46.875, 46.625, 46.375, 46.125, 45.875, 45.625, 45.375, 45.125],
          dtype='float64', name='lat'))

  • lon
    PandasIndex

    PandasIndex(Index([-4.875, -4.625, -4.375, -4.125, -3.875, -3.625, -3.375, -3.125, -2.875,
           -2.625,
           ...
           32.625, 32.875, 33.125, 33.375, 33.625, 33.875, 34.125, 34.375, 34.625,
           34.875],
          dtype='float64', name='lon', length=160))

  • time
    PandasIndex

    PandasIndex(DatetimeIndex(['2010-01-01', '2010-02-01', '2010-03-01', '2010-04-01',
                   '2010-05-01', '2010-06-01', '2010-07-01', '2010-08-01',
                   '2010-09-01', '2010-10-01', '2010-11-01', '2010-12-01'],
                  dtype='datetime64[ns]', name='time', freq=None))

• Attributes: (6)

  GRIB_centre :

  ecmf

  GRIB_centreDescription :

  European Centre for Medium-Range Weather Forecasts

  GRIB_subCentre :

  0

  Conventions :

  CF-1.7

  institution :

  European Centre for Medium-Range Weather Forecasts

  history :

  2024-09-18T08:40 GRIB to CDM+CF via cfgrib-0.9.14.0/ecCodes-2.36.0 with {"source": "data.grib", "filter_by_keys": {}, "encode_cf": ["parameter", "time", "geography", "vertical"]}

In [55]:

cds_dataset.attrs["title"] = "ERA5 2m Temperature"

cds_dataset.attrs["title"] = "ERA5 2m Temperature"

Let's set some attributes of the variable, so the colormapping does not fall back on the default

In [56]:

cds_dataset.t2m.attrs["color_value_min"] = 270
cds_dataset.t2m.attrs["color_value_max"] = 310
cds_dataset.t2m.attrs["color_bar_name"] = "plasma"

cds_dataset.t2m.attrs["color_value_min"] = 270 cds_dataset.t2m.attrs["color_value_max"] = 310 cds_dataset.t2m.attrs["color_bar_name"] = "plasma"

In [ ]:

viewer = Viewer()

viewer = Viewer()

In [58]:

viewer.add_dataset(dataset)
viewer.add_dataset(cds_dataset)

viewer.add_dataset(dataset) viewer.add_dataset(cds_dataset)

Out[58]:

'f6015205-89c6-49d5-a0c4-d7c90694d5b4'

You can click on the viewer link to open xcube Viewer in a new browser tab:

In [59]:

viewer.info()

viewer.info()

Server: https://deep.earthsystemdatalab.net/user/alicebalfanz/proxy/8006
Viewer: https://deep.earthsystemdatalab.net/user/alicebalfanz/proxy/8006/viewer/?serverUrl=https://deep.earthsystemdatalab.net/user/alicebalfanz/proxy/8006

You can also open xcube Viewer inlined here:

In [60]:

viewer.show()

viewer.show()

Out[60]:

To stop the server and viewer:

In [54]:

viewer.stop_server()

viewer.stop_server()

---

Scenario 3: Use custom server configuration to start server and pass it to the viewer constructor. In this case, we have created a local file with the configuration and load it as a dictionary and pass it to the viewer.

The custom configuration allows you to predefine your value ranges, the colormaps that should be used as well as which bands should be used to create an RGB image, then the RGB switch in the viewer will display the RGB image.

If you do not have a server-config.yaml file in your directory, please create one with the following content:

DataStores:
  - Identifier: deep-esdl-cci-sst
    StoreId: s3
    StoreParams:
      root: $S3_USER_STORAGE_BUCKET
      storage_options:
          anon: false
          key: $S3_USER_STORAGE_KEY
          secret: $S3_USER_STORAGE_SECRET
    Datasets:
      - Path: "*.zarr"
        Style: default

        # ChunkCacheSize: 1G


Styles:
  - Identifier: default
    ColorMappings:
      analysed_sst:
        ColorBar: plasma
        ValueRange: [270, 310]
## if you have bands that can create an RGB image, you can specify them as below. 
#      rgb:
#        Red:
#          Variable: B04
#          ValueRange: [0., 0.25]
#        Green:
#          Variable: B03
#          ValueRange: [0., 0.25]
#        Blue:
#          Variable: B02
#          ValueRange: [0., 0.25]

For all possible settings within the server configuration file, please checkout the documentation: https://xcube.readthedocs.io/en/latest/cli/xcube_serve.html#configuration-file

In [32]:

from xcube.util.config import load_configs

from xcube.util.config import load_configs

In [ ]:

viewer = Viewer(server_config=load_configs("server-config.yaml"))

viewer = Viewer(server_config=load_configs("server-config.yaml"))

In [36]:

viewer.show()

viewer.show()

Out[36]:

To stop the server and viewer:

In [37]:

viewer.stop_server()

viewer.stop_server()

Afer all our testing and exploring, let's clean up the example cube :)

In [38]:

team_store.delete_data(output_id)

team_store.delete_data(output_id)

In [ ]: