Note

Download full notebook here

Model Formating Overview

This notebook was created to enable common formatting for the Early Release Science modeling initiatives. We will review:

1. Variable terminology

2. File naming schemes

3. Data formating

4. Physical unit archiving

This is namely for the booking of the following model types:

1. 1D-3D climate (spatial- and/or altitude- dependent climate)

2. 1D-3D chemistry (spatial- and/or altitude- dependent atmospheric composition)

3. 1D-3D cloud (spatial- and/or altitude- dependent single scattering, asymmetries, cloud optical depth)

4. Spectroscopy (flux, transit depth as a function of wavelength)

However, it can be applied to other modeling products (e.g. mixing profiles).

Variable Terminology

All file names and meta data should conform to the following variable names. Note, that these will not apply to all models. This is just an initial list. Please shoot me a DM, or slack for additional parameters to add (natasha.e.batalha@nasa.gov, or ERS slack channel)

Planet parameters (planet_params)

1. rp: planet radius

2. mp: planet mass

3. tint: object internal temperature

4. heat_redis: heat redistribution (only relevant for irradiated objects)

5. p_reference: reference pressure radius

6. pteff: planetary effective temperature

7. mh : metallicity

8. cto : carbon to oxygen ratio

9. logkzz : log of the kzz eddy diffusion

Stellar parameters (stellar_params)

1. logg : gravity

2. feh : stellar metallicity

3. steff : stellar effective temperature

4. rs : stellar radius

5. ms : stellar mass

Orbital parameters (orbit_params)

1. sma : semi-major axis

Cloud parameters (cld_params)

1. opd : extinction optical depth

2. ssa : single scattering albedo

3. asy : asymmetry parameter

4. fsed : cloud sedimentation efficiency parameter

Model Gridding ( coords)

1. pressure: pressure grid

2. wavelength: wavelength grid

3. wno: wavenumber grid

4. lat: latitude grid

5. lon: longitude grid

Model Output ( data_vars)

There are SO many different model outputs users will want to pass. For the purposes of ERS, we will focus on these, but feel free to send recommendations for more. Note that in your xarray file there will not be a separation between these categories. They will all be lumped into data_vars. However their coordinate systems will be different! The beauty of xarray!

Spectrum

1. transit_depth : transmission spectrum reported as unitless depth (rp/rs)^2. This way it can be directly compared to data.

2. fpfs_emission : relative emission spectrum (unitless)

3. fpfs_reflection relative reflected light spectrum (unitless)

4. flux_emission : thermal emission in raw flux units

5. albedo : albedo spectrum

Chemistry

6. case sensitive molecule names (e.g. Na, H2O, TiO) for each chemical abundance (either 1d or 3d). This means your mixing ratio profile for TiO would not be TIO. Or, for example the chemical profile for sodium would be “Na” not NA

Climate

7. temperature: computed temperature profile either 1d or 3d

Cloud

8. opd : extinction optical depth

9. ssa : single scattering albedo

10. asy : asymmetry parameter

Retrieval output

11. Coming soon.

Specifying units

We should be able to convert all units to astropy.units. For unitless parameters (e.g. single scattering albedo, optical depth) unitless designation should be provided. See example:

import astropy.units as u

#examples of valid units

u.Unit('cm') #Valid
#u.Unit('CM') #NOT valid
u.Unit("R_jup")#Valid
u.Unit("R_jupiter")#Valid
#u.Unit("R_Jupiter")#NOT Valid

$\mathrm{R_{\rm J}}$

unit = 'cm'
#doing it this away enables easy conversions. for example:
(1*u.Unit('R_jup')).to('cm')

$7.1492 \times 10^{9} \; \mathrm{cm}$

Data Types: Using xarray

xarray: N-D labeled arrays and datasets in Python: From their website: “array introduces labels in the form of dimensions, coordinates and attributes on top of raw NumPy-like arrays, which allows for a more intuitive, more concise, and less error-prone developer experience. The package includes a large and growing library of domain-agnostic functions for advanced analytics and visualization with these data structures.”

Xarray is your friend and will make it very easy for other folks to use your data. Let’s build some simple examples.

This is how we make sure we credit the appropriate people for their work, and understand the meta data surrounding the model runs.

1. author: Author or author list

2. contact email: point of contact

3. code used: (can be a dictionary. e.g. {‘chemistry’:’vulcan’, ‘cloud’:’virga’, ‘spectra’:’chimera’}

1. doi (str): made sure to include if you want your work referenced!

2. planet_params (json dict) : with dict keys defined in section 1

3. stellar_params (json dict): with dict keys defined in section 1

4. orbit_params (json dict) : with dict keys defined in section 1

5. cld_params (json dict): with dict keys defined in section 1

6. model_notes (str) : any additional modeling notes that you want the user to be aware of

Easy Example: 1D data: e.g. P-T profiles, chemistry

Here we will show an example with pressure as the dependent variable. Spectra, which are on a wavelength or wavenumber grid, can also be stored similarly.

import numpy as np
import xarray as xr
from astropy.utils.misc import JsonCustomEncoder
import json #we will use this to dump model parameters into an attribute

#fake pressure temperature profile at 1000 K
pressure = np.logspace(-6,2,50)
temperature = np.logspace(-6,2,50)*0 + 1000

Practice convert to xarray. In this case we are storing temperature data, labeled with unit “Kelvin” that is on a grid of pressure with units of “bar”

# put data into a dataset where each
ds = xr.Dataset(
    data_vars=dict(
        temperature=(["pressure"], temperature,{'units': 'Kelvin'})#, required
    ),
    coords=dict(
        pressure=(["pressure"], pressure,{'units': 'bar'})#required*
    ),
    attrs=dict(author="NE Batalha",#required
               contact="natasha.e.batalha@nasa.gov",#required
               code="numpy", #required, in this case I used numpy to make my fake model.
               doi="add your paper here",#optional if there is a citation to reference
               planet_params=json.dumps({'rp':1*u.Unit('R_jup'), 'mp':1*u.Unit('M_jup')},
                                        cls=JsonCustomEncoder), #optional in accordance with model runs
               stellar_params=json.dumps({'rs':1*u.Unit('R_sun'), 'ms':1*u.Unit('M_sun')},
                                         cls=JsonCustomEncoder) #optional in accordance with model runs
              ),
)

#printing is easy
ds

<xarray.Dataset> Size: 800B
Dimensions:      (pressure: 50)
Coordinates:
  * pressure     (pressure) float64 400B 1e-06 1.456e-06 ... 68.66 100.0
Data variables:
    temperature  (pressure) float64 400B 1e+03 1e+03 1e+03 ... 1e+03 1e+03 1e+03
Attributes:
    author:          NE Batalha
    contact:         natasha.e.batalha@nasa.gov
    code:            numpy
    doi:             add your paper here
    planet_params:   {"rp": {"value": 1.0, "unit": "jupiterRad"}, "mp": {"val...
    stellar_params:  {"rs": {"value": 1.0, "unit": "solRad"}, "ms": {"value":...

xarray.Dataset

• Dimensions:
  • pressure: 50
• Coordinates: (1)
  • pressure
    (pressure)
    float64
    1e-06 1.456e-06 ... 68.66 100.0

    units :

    bar

    array([1.000000e-06, 1.456348e-06, 2.120951e-06, 3.088844e-06, 4.498433e-06,
           6.551286e-06, 9.540955e-06, 1.389495e-05, 2.023590e-05, 2.947052e-05,
           4.291934e-05, 6.250552e-05, 9.102982e-05, 1.325711e-04, 1.930698e-04,
           2.811769e-04, 4.094915e-04, 5.963623e-04, 8.685114e-04, 1.264855e-03,
           1.842070e-03, 2.682696e-03, 3.906940e-03, 5.689866e-03, 8.286428e-03,
           1.206793e-02, 1.757511e-02, 2.559548e-02, 3.727594e-02, 5.428675e-02,
           7.906043e-02, 1.151395e-01, 1.676833e-01, 2.442053e-01, 3.556480e-01,
           5.179475e-01, 7.543120e-01, 1.098541e+00, 1.599859e+00, 2.329952e+00,
           3.393222e+00, 4.941713e+00, 7.196857e+00, 1.048113e+01, 1.526418e+01,
           2.222996e+01, 3.237458e+01, 4.714866e+01, 6.866488e+01, 1.000000e+02])

• Data variables: (1)
  • temperature
    (pressure)
    float64
    1e+03 1e+03 1e+03 ... 1e+03 1e+03

    units :

    Kelvin

    array([1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000.,
           1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000.,
           1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000.,
           1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000.,
           1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000., 1000.,
           1000., 1000., 1000., 1000., 1000.])

• Indexes: (1)
  • pressure
    PandasIndex

    PandasIndex(Index([                 1e-06, 1.4563484775012444e-06, 2.1209508879201924e-06,
           3.0888435964774785e-06,  4.498432668969444e-06,  6.551285568595509e-06,
            9.540954763499944e-06,  1.389495494373136e-05, 2.0235896477251556e-05,
             2.94705170255181e-05,  4.291934260128778e-05,  6.250551925273976e-05,
            9.102981779915228e-05,  0.0001325711365590108, 0.00019306977288832496,
            0.0002811768697974228,  0.0004094915062380423,  0.0005963623316594642,
             0.000868511373751352,  0.0012648552168552957,  0.0018420699693267144,
            0.0026826957952797246,   0.003906939937054613,   0.005689866029018294,
             0.008286427728546842,   0.012067926406393288,   0.017575106248547894,
             0.025595479226995333,    0.03727593720314938,    0.05428675439323859,
              0.07906043210907684,    0.11513953993264457,    0.16768329368110066,
                0.244205309454865,    0.35564803062231287,     0.5179474679231202,
               0.7543120063354607,     1.0985411419875573,     1.5998587196060572,
               2.3299518105153667,      3.393221771895323,      4.941713361323828,
                7.196856730011514,     10.481131341546853,     15.264179671752302,
                22.22996482526191,       32.3745754281764,       47.1486636345739,
                68.66488450042998,                  100.0],
          dtype='float64', name='pressure'))

• Attributes: (6)

  author :

  NE Batalha

  contact :

  natasha.e.batalha@nasa.gov

  code :

  numpy

  doi :

  add your paper here

  planet_params :

  {"rp": {"value": 1.0, "unit": "jupiterRad"}, "mp": {"value": 1.0, "unit": "jupiterMass"}}

  stellar_params :

  {"rs": {"value": 1.0, "unit": "solRad"}, "ms": {"value": 1.0, "unit": "solMass"}}

#plotting is easy
ds['temperature'].plot(y='pressure')

[<matplotlib.lines.Line2D at 0x7fe946276290>]

Looping to add many variables to data_vars

If you have a big table of many parameters, e.g. molecules you can pre-make data_vars as a dictionary then add it like this:

fake_chemistry = {i:np.random.randn(len(pressure)) for i in ['H2O','CH4','CO2','CO',"H2S"]}

data_vars=dict(
        temperature=(["pressure"], temperature,{'units': 'Kelvin'})#, required
    )
for i in fake_chemistry.keys():
    data_vars[i] = (["pressure"], fake_chemistry[i],{'units': 'v/v'})#volume mixing ratio units

# put data into a dataset where each
ds = xr.Dataset(
    data_vars=data_vars,
    coords=dict(
        pressure=(["pressure"], pressure,{'units': 'bar'})#required*
    ),
    attrs=dict(author="NE Batalha",#required
               contact="natasha.e.batalha@nasa.gov",#required
               code=json.dumps({'climate':"numpy","chemistry":'numpy'}), #required
               doi="",#optional if there is a citation to reference
               planet_params=json.dumps({'rp':1*u.Unit('R_jup'), 'mp':1*u.Unit('M_jup')},
                                        cls=JsonCustomEncoder), #optional in accordance with model runs
               stellar_params=json.dumps({'rs':1*u.Unit('R_sun'), 'ms':1*u.Unit('M_sun')},
                                         cls=JsonCustomEncoder) #optional in accordance with model runs #optional in accordance with model runs
              ),
)

2D data: e.g. cloud profiles with pressure vs wavenumber

wno = np.linspace(10,10000,400)#fake wavenumber array
opd = np.zeros((len(pressure), len(wno))) + 1
ssa = np.zeros((len(pressure), len(wno))) + 0.9
asy = np.zeros((len(pressure), len(wno))) + 0.8

# put data into a dataset where each
ds = xr.Dataset(
    #now data is a function of two dimensions
    data_vars=dict(opd=(["pressure","wno"], opd,{'units': 'unitless per layer'}),
                   ssa=(["pressure","wno"], ssa,{'units': 'unitless'}),
                   asy=(["pressure","wno"], asy,{'units': 'unitless'}),
                  ),
    coords=dict(
        pressure=(["pressure"], pressure,{'units': 'bar'}),#required
        wno=(["wno"], wno,{'units': 'cm**(-1)'})#required
    ),
    attrs=dict(author="NE Batalha",#required
               contact="natasha.e.batalha@nasa.gov",#required
               code='numpy', #required
               doi="",#optional if there is a citation to reference
               planet_params=json.dumps({'rp':1*u.Unit('R_jup'), 'mp':1*u.Unit('M_jup')},
                                        cls=JsonCustomEncoder), #optional in accordance with model runs
               stellar_params=json.dumps({'rs':1*u.Unit('R_sun'), 'ms':1*u.Unit('M_sun')},
                                         cls=JsonCustomEncoder) #optional in accordance with model runs #optional in accordance with model runs
              ),
)

3D data: e.g. GCM pressure grid

xarray is incredibly useful for GCM work. Here is an example of how picaso uses xarray and xesfm to do regridding and 3d calculations.

Here we’ll just show how to create a 3D gridded dataset.

lat = np.linspace(-90,85,40)#fake latitude array
lon = np.linspace(-180,175,100)#fake latitude array
temperature_3d = np.random.randn(len(lat),len(lon),len(pressure))

# put data into a dataset where each
ds = xr.Dataset(
    #now data is a function of two dimensions
    data_vars=dict(temperature=(["lat","lon","pressure"],temperature_3d ,{'units': 'Kelvin'})
                  ),
    coords=dict(
        pressure=(["pressure"], pressure,{'units': 'bar'}),#required
        lat=(["lat"], lat,{'units': 'degree'}),#required
        lon=(["lon"], lon,{'units': 'degree'}),#required
    ),
    attrs=dict(author="NE Batalha",#required
               contact="natasha.e.batalha@nasa.gov",#required
               code='numpy', #required
               doi="",#optional if there is a citation to reference
               planet_params=json.dumps({'rp':1*u.Unit('R_jup'), 'mp':1*u.Unit('M_jup')},
                                        cls=JsonCustomEncoder), #optional in accordance with model runs
               stellar_params=json.dumps({'rs':1*u.Unit('R_sun'), 'ms':1*u.Unit('M_sun')},
                                         cls=JsonCustomEncoder) #optional in accordance with model runs #optional in accordance with model runs
              ),
)

#easy plotting
ds['temperature'].isel(pressure=10).plot()

<matplotlib.collections.QuadMesh at 0x7fe946342550>

Storing xarray data

Filenaming

We usually rely on a long filename to give us information about the model. If we properly use attrs then filenaming does not matter. However, friendly filenames are always appreciated by people using your models. We suggest the following naming convention.

Given independent variables (x,y,z): tag_x{x}_y{y}_z{z}.nc

For example: jupiter_mh1_teff1000_tint100.nc

Using netcdf

“The recommended way to store xarray data structures is netCDF, which is a binary file format for self-described datasets that originated in the geosciences. Xarray is based on the netCDF data model, so netCDF files on disk directly correspond to Dataset objects (more accurately, a group in a netCDF file directly corresponds to a Dataset object. See Groups for more.)” - Quoted from xarray website

ds.to_netcdf("/data/picaso_dbs/fakeplanet_1000teq.nc")

Using pickle

import pickle as pk
pk.dump(ds, open("/data/picaso_dbs/fakeplanet_1000teq.pk",'wb'))

Reading/interpreting an xarray file

First, make sure you have installed netCDF4 and h5netcdf :

pip install netCDF4
pip install h5netcdf

or if you prefer conda

conda install -c conda-forge netCDF4

ds_sm = xr.open_dataset("profile_eq_planet_300_grav_4.5_mh_+2.0_CO_2.0_sm_0.0486_v_0.5_.nc")

Look at all the information we can glean from this

ds_sm #39 data variables

<xarray.Dataset> Size: 52kB
Dimensions:        (pressure: 91, wavelength: 1498)
Coordinates:
  * pressure       (pressure) float64 728B 1e-06 1.237e-06 ... 161.7 200.0
  * wavelength     (wavelength) float64 12kB 5.994 5.982 5.97 ... 0.3009 0.3003
Data variables: (12/39)
    temperature    (pressure) float64 728B ...
    e-             (pressure) float64 728B ...
    H2             (pressure) float64 728B ...
    H              (pressure) float64 728B ...
    H+             (pressure) float64 728B ...
    H-             (pressure) float64 728B ...
    ...             ...
    OCS            (pressure) float64 728B ...
    Li             (pressure) float64 728B ...
    LiOH           (pressure) float64 728B ...
    LiH            (pressure) float64 728B ...
    LiCl           (pressure) float64 728B ...
    transit_depth  (wavelength) float64 12kB ...
Attributes:
    author:          Sagnick Mukherjee
    contact:         samukher@ucsc.edu
    code:            {"spectrum": "PICASO", "climate": "PICASO", "chemistry":...
    doi:             Mukherjee et al. (Submitted)
    planet_params:   {"rp": {"value": 1.279, "unit": "jupiterRad"}, "mp": {"v...
    stellar_params:  {"rs": {"value": 0.932, "unit": "solRad"}, "logg": 4.389...
    orbit_params:    {"sma": 0.0486}
    cld_params:      {"opd": "None", "ssa": "None", "asy": "None", "p_cloud":...

xarray.Dataset

• Dimensions:
  • pressure: 91
  • wavelength: 1498
• Coordinates: (2)
  • pressure
    (pressure)
    float64
    1e-06 1.237e-06 ... 161.7 200.0

    units :

    bar

    array([1.000000e-06, 1.236613e-06, 1.529211e-06, 1.891041e-06, 2.338485e-06,
           2.891801e-06, 3.576037e-06, 4.422172e-06, 5.468514e-06, 6.762433e-06,
           8.362510e-06, 1.034119e-05, 1.278804e-05, 1.581385e-05, 1.955561e-05,
           2.418271e-05, 2.990465e-05, 3.698046e-05, 4.573051e-05, 5.655092e-05,
           6.993158e-05, 8.647827e-05, 1.069401e-04, 1.322435e-04, 1.635340e-04,
           2.022282e-04, 2.500779e-04, 3.092495e-04, 3.824218e-04, 4.729076e-04,
           5.848035e-04, 7.231754e-04, 8.942879e-04, 1.105888e-03, 1.367555e-03,
           1.691135e-03, 2.091279e-03, 2.586102e-03, 3.198006e-03, 3.954695e-03,
           4.890426e-03, 6.047562e-03, 7.478491e-03, 9.247997e-03, 1.143619e-02,
           1.414214e-02, 1.748834e-02, 2.162631e-02, 2.674336e-02, 3.307118e-02,
           4.089624e-02, 5.057280e-02, 6.253896e-02, 7.733647e-02, 9.563525e-02,
           1.182638e-01, 1.462464e-01, 1.808502e-01, 2.236416e-01, 2.765581e-01,
           3.419952e-01, 4.229156e-01, 5.229827e-01, 6.467270e-01, 7.997508e-01,
           9.889819e-01, 1.222987e+00, 1.512362e+00, 1.870206e+00, 2.312720e+00,
           2.859938e+00, 3.536636e+00, 4.373448e+00, 5.408261e+00, 6.687924e+00,
           8.270371e+00, 1.022725e+01, 1.264714e+01, 1.563961e+01, 1.934014e+01,
           2.391626e+01, 2.957515e+01, 3.657301e+01, 4.522664e+01, 5.592783e+01,
           6.916106e+01, 8.552544e+01, 1.057618e+02, 1.307864e+02, 1.617321e+02,
           2.000000e+02])

  • wavelength
    (wavelength)
    float64
    5.994 5.982 5.97 ... 0.3009 0.3003

    units :

    micron

    array([5.993991, 5.982015, 5.970063, ..., 0.301528, 0.300925, 0.300324])

• Data variables: (39)
  • temperature
    (pressure)
    float64
    ...

    units :

    Kelvin

    [91 values with dtype=float64]

  • e-
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H2
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H+
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H-
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H2-
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H2+
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H3+
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • He
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H2O
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • CH4
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • CO
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • NH3
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • N2
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • PH3
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • H2S
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • TiO
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • VO
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • Fe
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • FeH
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • CrH
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • Na
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • K
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • Rb
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • Cs
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • CO2
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • HCN
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • C2H2
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • C2H4
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • C2H6
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • SiO
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • MgH
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • OCS
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • Li
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • LiOH
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • LiH
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • LiCl
    (pressure)
    float64
    ...

    units :

    v/v

    [91 values with dtype=float64]

  • transit_depth
    (wavelength)
    float64
    ...

    units :

    (R_jup*R_jup)/(R_jup*R_jup)

    [1498 values with dtype=float64]

• Indexes: (2)
  • pressure
    PandasIndex

    PandasIndex(Index([                 1e-06,  1.236612598156165e-06, 1.5292107179185409e-06,
           1.8910412390135015e-06, 2.3384854197969397e-06, 2.8918005307254037e-06,
           3.5760369676497184e-06,  4.422172365667813e-06,  5.468514058602868e-06,
            6.762433378062408e-06,  8.362510309503727e-06, 1.0341185600943119e-05,
           1.2788040393997395e-05,  1.581385185694711e-05, 1.9555608431676103e-05,
            2.418271175121959e-05,  2.990464600913728e-05,  3.698046199829965e-05,
            4.573050519273266e-05,  5.655091884137913e-05,  6.993157867655628e-05,
            8.647827120037853e-05, 0.00010694011963315357, 0.00013224349918668514,
            0.0001635339771185094, 0.00020222817633133082,    0.00025007791055347,
           0.00030924949471099195,  0.0003824218211330409, 0.00047290764182294197,
            0.0005848035476425735,  0.0007231754344612254,  0.0008942878529318094,
            0.0011058876253135033,  0.0013675545696076828,  0.0016911352094428928,
             0.002091279105182546,   0.002586102087729489,   0.003198006421804249,
            0.0039546950301874535,  0.0048904256961953795,   0.006047562026261841,
             0.007478491389806217,   0.009247996667836777,   0.011436189187153192,
              0.01414213562373095,   0.017488343077138792,   0.021626305370066986,
              0.02674336167219716,   0.033071177960885725,     0.0408962353022958,
             0.050572799791977895,    0.06253896134678935,    0.07733646747704116,
              0.09563524997900384,    0.11826375495185025,    0.14624644927871158,
              0.18085020161366133,      0.223641637694536,    0.27655806664533994,
               0.3419951893353397,      0.422915559640884,     0.5229827090081821,
               0.6467270065773577,     0.7997507639013856,     0.9889818700254702,
               1.2229874398215397,     1.5123616754700706,     1.8702055008548548,
                2.312719683498074,     2.8599382966174574,       3.53663572754643,
                4.373448295773112,      5.408261259937639,      6.687924008158817,
                8.270371084000287,     10.227245073901214,      12.64714010281682,
               15.639612781789337,      19.34014219624488,     23.916263490008067,
               29.575152732566306,      36.57300646148422,      45.22664054271822,
               55.927833467405726,      69.16106345337391,      85.52544236832011,
               105.76183949554371,      130.7864231243596,      161.7321385033659,
               200.00000000000003],
          dtype='float64', name='pressure'))

  • wavelength
    PandasIndex

    PandasIndex(Index([  5.993990916322709,   5.982014910495892,   5.970062832752643,
            5.9581346352846065,   5.946230270378943,   5.934349690418148,
              5.92249284787985,   5.910659695336634,   5.898850185455842,
             5.887064270999387,
           ...
           0.30577880497031257,  0.3051678583070352, 0.30455813231641177,
            0.3039496245595358, 0.30334233260237387,  0.3027362540157557,
            0.3021313863753647, 0.30152772726172755, 0.30092527426020566,
           0.30032402496098454],
          dtype='float64', name='wavelength', length=1498))

• Attributes: (8)

  author :

  Sagnick Mukherjee

  contact :

  samukher@ucsc.edu

  code :

  {"spectrum": "PICASO", "climate": "PICASO", "chemistry": "PICASO"}

  doi :

  Mukherjee et al. (Submitted)

  planet_params :

  {"rp": {"value": 1.279, "unit": "jupiterRad"}, "mp": {"value": 0.28, "unit": "jupiterMass"}, "tint": 300.0, "heat_redis": 0.5, "mh": 100.0, "cto": 0.916, "rainout": "None", "p_quench": "None", "p_reference": {"value": 10.0, "unit": "bar"}, "logkzz": "None"}

  stellar_params :

  {"rs": {"value": 0.932, "unit": "solRad"}, "logg": 4.38933, "steff": 5326.6, "feh": -0.03, "ms": {"value": 0.913, "unit": "solMass"}}

  orbit_params :

  {"sma": 0.0486}

  cld_params :

  {"opd": "None", "ssa": "None", "asy": "None", "p_cloud": "None", "haze_effec": "None", "fsed": "None"}

ds_sm['wavelength']#data operates very similarly to pandas, note we can see the unit of the coordinate system

<xarray.DataArray 'wavelength' (wavelength: 1498)> Size: 12kB
array([5.993991, 5.982015, 5.970063, ..., 0.301528, 0.300925, 0.300324])
Coordinates:
  * wavelength  (wavelength) float64 12kB 5.994 5.982 5.97 ... 0.3009 0.3003
Attributes:
    units:    micron

xarray.DataArray

'wavelength'

• wavelength: 1498

• 5.994 5.982 5.97 5.958 5.946 ... 0.3027 0.3021 0.3015 0.3009 0.3003

  array([5.993991, 5.982015, 5.970063, ..., 0.301528, 0.300925, 0.300324])

• Coordinates: (1)
  • wavelength
    (wavelength)
    float64
    5.994 5.982 5.97 ... 0.3009 0.3003

    units :

    micron

    array([5.993991, 5.982015, 5.970063, ..., 0.301528, 0.300925, 0.300324])

• Indexes: (1)
  • wavelength
    PandasIndex

    PandasIndex(Index([  5.993990916322709,   5.982014910495892,   5.970062832752643,
            5.9581346352846065,   5.946230270378943,   5.934349690418148,
              5.92249284787985,   5.910659695336634,   5.898850185455842,
             5.887064270999387,
           ...
           0.30577880497031257,  0.3051678583070352, 0.30455813231641177,
            0.3039496245595358, 0.30334233260237387,  0.3027362540157557,
            0.3021313863753647, 0.30152772726172755, 0.30092527426020566,
           0.30032402496098454],
          dtype='float64', name='wavelength', length=1498))

• Attributes: (1)

  units :

  micron

ds_sm['wavelength'].values #same as pandas!

array([5.99399092, 5.98201491, 5.97006283, ..., 0.30152773, 0.30092527,
       0.30032402])

ds_sm['temperature']

<xarray.DataArray 'temperature' (pressure: 91)> Size: 728B
[91 values with dtype=float64]
Coordinates:
  * pressure  (pressure) float64 728B 1e-06 1.237e-06 1.529e-06 ... 161.7 200.0
Attributes:
    units:    Kelvin

xarray.DataArray

'temperature'

• pressure: 91

• ...

  [91 values with dtype=float64]

• Coordinates: (1)
  • pressure
    (pressure)
    float64
    1e-06 1.237e-06 ... 161.7 200.0

    units :

    bar

    array([1.000000e-06, 1.236613e-06, 1.529211e-06, 1.891041e-06, 2.338485e-06,
           2.891801e-06, 3.576037e-06, 4.422172e-06, 5.468514e-06, 6.762433e-06,
           8.362510e-06, 1.034119e-05, 1.278804e-05, 1.581385e-05, 1.955561e-05,
           2.418271e-05, 2.990465e-05, 3.698046e-05, 4.573051e-05, 5.655092e-05,
           6.993158e-05, 8.647827e-05, 1.069401e-04, 1.322435e-04, 1.635340e-04,
           2.022282e-04, 2.500779e-04, 3.092495e-04, 3.824218e-04, 4.729076e-04,
           5.848035e-04, 7.231754e-04, 8.942879e-04, 1.105888e-03, 1.367555e-03,
           1.691135e-03, 2.091279e-03, 2.586102e-03, 3.198006e-03, 3.954695e-03,
           4.890426e-03, 6.047562e-03, 7.478491e-03, 9.247997e-03, 1.143619e-02,
           1.414214e-02, 1.748834e-02, 2.162631e-02, 2.674336e-02, 3.307118e-02,
           4.089624e-02, 5.057280e-02, 6.253896e-02, 7.733647e-02, 9.563525e-02,
           1.182638e-01, 1.462464e-01, 1.808502e-01, 2.236416e-01, 2.765581e-01,
           3.419952e-01, 4.229156e-01, 5.229827e-01, 6.467270e-01, 7.997508e-01,
           9.889819e-01, 1.222987e+00, 1.512362e+00, 1.870206e+00, 2.312720e+00,
           2.859938e+00, 3.536636e+00, 4.373448e+00, 5.408261e+00, 6.687924e+00,
           8.270371e+00, 1.022725e+01, 1.264714e+01, 1.563961e+01, 1.934014e+01,
           2.391626e+01, 2.957515e+01, 3.657301e+01, 4.522664e+01, 5.592783e+01,
           6.916106e+01, 8.552544e+01, 1.057618e+02, 1.307864e+02, 1.617321e+02,
           2.000000e+02])

• Indexes: (1)
  • pressure
    PandasIndex

    PandasIndex(Index([                 1e-06,  1.236612598156165e-06, 1.5292107179185409e-06,
           1.8910412390135015e-06, 2.3384854197969397e-06, 2.8918005307254037e-06,
           3.5760369676497184e-06,  4.422172365667813e-06,  5.468514058602868e-06,
            6.762433378062408e-06,  8.362510309503727e-06, 1.0341185600943119e-05,
           1.2788040393997395e-05,  1.581385185694711e-05, 1.9555608431676103e-05,
            2.418271175121959e-05,  2.990464600913728e-05,  3.698046199829965e-05,
            4.573050519273266e-05,  5.655091884137913e-05,  6.993157867655628e-05,
            8.647827120037853e-05, 0.00010694011963315357, 0.00013224349918668514,
            0.0001635339771185094, 0.00020222817633133082,    0.00025007791055347,
           0.00030924949471099195,  0.0003824218211330409, 0.00047290764182294197,
            0.0005848035476425735,  0.0007231754344612254,  0.0008942878529318094,
            0.0011058876253135033,  0.0013675545696076828,  0.0016911352094428928,
             0.002091279105182546,   0.002586102087729489,   0.003198006421804249,
            0.0039546950301874535,  0.0048904256961953795,   0.006047562026261841,
             0.007478491389806217,   0.009247996667836777,   0.011436189187153192,
              0.01414213562373095,   0.017488343077138792,   0.021626305370066986,
              0.02674336167219716,   0.033071177960885725,     0.0408962353022958,
             0.050572799791977895,    0.06253896134678935,    0.07733646747704116,
              0.09563524997900384,    0.11826375495185025,    0.14624644927871158,
              0.18085020161366133,      0.223641637694536,    0.27655806664533994,
               0.3419951893353397,      0.422915559640884,     0.5229827090081821,
               0.6467270065773577,     0.7997507639013856,     0.9889818700254702,
               1.2229874398215397,     1.5123616754700706,     1.8702055008548548,
                2.312719683498074,     2.8599382966174574,       3.53663572754643,
                4.373448295773112,      5.408261259937639,      6.687924008158817,
                8.270371084000287,     10.227245073901214,      12.64714010281682,
               15.639612781789337,      19.34014219624488,     23.916263490008067,
               29.575152732566306,      36.57300646148422,      45.22664054271822,
               55.927833467405726,      69.16106345337391,      85.52544236832011,
               105.76183949554371,      130.7864231243596,      161.7321385033659,
               200.00000000000003],
          dtype='float64', name='pressure'))

• Attributes: (1)

  units :

  Kelvin

How to get attributes from string dictionary

json.loads(ds_sm.attrs['planet_params'])

{'rp': {'value': 1.279, 'unit': 'jupiterRad'},
 'mp': {'value': 0.28, 'unit': 'jupiterMass'},
 'tint': 300.0,
 'heat_redis': 0.5,
 'mh': 100.0,
 'cto': 0.916,
 'rainout': 'None',
 'p_quench': 'None',
 'p_reference': {'value': 10.0, 'unit': 'bar'},
 'logkzz': 'None'}

json.loads(ds_sm.attrs['planet_params'])['rp'] #radius used

{'value': 1.279, 'unit': 'jupiterRad'}

Checking your data is in compliance

TLDR: this function will check that your data can be properly interpretted

def data_check(usr_xa):
    """This function will check that all the requirements have been met"""

    #step 1: check that required attributes are present
    assert 'author' in usr_xa.attrs ,'No author information in attrs'
    assert 'contact' in usr_xa.attrs ,'No contact information in attrs'
    assert 'code' in usr_xa.attrs , 'Code used was not specified in attrs'

    #step 2: check that all coordinates have units
    try:
        for i in usr_xa.coords.keys(): test= usr_xa[i].units
    except AttributeError:
        print(f'Missing unit for {i} coords')

    #step 2: check that all coordinates have units
    try:
        for i in usr_xa.data_vars.keys(): test=usr_xa[i].units
    except AttributeError:
        print(f'Missing unit for {i} data_var')

    #step 3: check that some attrs is a proper dictionary
    try :
        for i in usr_xa.attrs:
            #these need to be dictionaries to be interpretable
            if i in ['planet_params','stellar_params','cld_params','orbit_params']:
                json.loads(usr_xa.attrs[i])
    except ValueError:
        print(f"Was not able to read attr for {i}. This means that you did not properly define a dictionary with json and a dict."," For example: json.dumps({'mp':1,'rp':1})")

    #step 4: hurray if you have made it to here this is great
    #last thing is the least important -- to make sure that we agree on terminology
    for i in usr_xa.attrs:
        if i == 'planet_params':
            for model_key in json.loads(usr_xa.attrs[i]).keys():
                assert model_key in ['rp', 'mp', 'tint', 'heat_redis', 'p_reference','rainout','p_quench',
                'pteff', 'mh' , 'cto' , 'logkzz'], f'Could not find {model_key} in listed planet_params attr. This might be because we havent added it yet! Check your terms and contact us if this is the case'

        elif  i == 'stellar_params':
            for model_key in json.loads(usr_xa.attrs[i]).keys():
                assert model_key in ['logg', 'feh', 'steff', 'rs', 'ms',
                ], f'Could not find {model_key} in listed stellar_params attr. This might be because we havent added it yet! Check your terms and contact us if this is the case'

        elif  i == 'orbit_params':
            for model_key in json.loads(usr_xa.attrs[i]).keys():
                assert model_key in ['sma',
                ], f'Could not find {model_key} in listed orbit_params attr. This might be because we havent added it yet! Check your terms and contact us if this is the case'

        elif  i == 'cld_params':
            for model_key in json.loads(usr_xa.attrs[i]).keys():
                assert model_key  in ['opd','ssa','asy','fsed','p_cloud','haze_effec',
                ], f'Could not find {model_key} in listed cld_params attr. This might be because we havent added it yet! Check your terms and contact us if this is the case'

    print('SUCCESS!')

ds_sm = xr.open_dataset("profile_eq_planet_300_grav_4.5_mh_+2.0_CO_2.0_sm_0.0486_v_0.5_.nc")
data_check(ds_sm)

SUCCESS!