ERA5 dataset

Post questions and find resources to convert meteorological data into a format HYSPLIT can read.
charleshelms
Posts: 2
Joined: December 9th, 2020, 12:20 pm
Registered HYSPLIT User: Yes

Re: ERA5 dataset

Post by charleshelms »

Hi all,
Has anyone managed to get era52arl working when downloading from "new" (circa 2018) Copernicus system? I had successfully converted ERA5 data to HYSPLIT when I downloaded ERA5 from the old MARS system circa 2017. The error I get is the same "latitudeOfFirstGridPointInDegrees" error that is discussed in this thread. Looking in the grib files, it appears that the variable exists in the files. I've included the python code I used to retrieve one of the files below:

#!/MYPATH/anaconda3/bin/python
import cdsapi
c = cdsapi.Client()
c.retrieve('reanalysis-era5-complete', {
'class': 'ea',
'date':'2013-09-05',
'expver': '1',
'levelist': '10/20/30/50/70/100/125/150/175/200/225/250/300/350/400/450/500/550/600/650/700/750/775/800/825/850/875/900/925/950/975/1000',
'levtype': 'pl',
'param': '60.128/75.128/76.128/129.128/130.128/131/132/133.128/135.128/138.128/155.128/157.128/203.128/246.128/247.128/248.128',
'stream': 'oper',
'time': '00:00:00',
'type': 'an',
'area': [50, -105, 0, -10],
'format':'grib'
}, 'era5_atlantic_201309050000')

In theory, I suppose I could go back to using the MARS system, but it seems like the Copernicus system is the preferred way to retrieve the data, so I'd like to get this working with that data if at all possible.

Thanks in advance,
Chip Helms
alicec
Posts: 260
Joined: February 8th, 2016, 12:56 pm
Registered HYSPLIT User: Yes

Re: ERA5 dataset

Post by alicec »

Can you make sure that you are using the most up to date era52arl code?
You can get it from github as well as the lastest HYSPLIT release.
https://github.com/amcz/hysplit_metdata

I have also added some sample code in the README.
For the pressure level data I am using the 'reanalysis-era5-pressure-levels'
rather than 'reanalysis-era5-complete' although I need to use the latter to get the ensemble data
and it should work for the pressure level data as well.

Can you add the code you are using to get the surface data
and the command for the conversion?
alicec
Posts: 260
Joined: February 8th, 2016, 12:56 pm
Registered HYSPLIT User: Yes

Re: ERA5 dataset

Post by alicec »

I reproduced your problem by downloading files using the request code that you have posted.
It looks like your request results in a file with
gridType = sh
please modify your request so
gridType = regular_ll

See some examples of requests at
https://github.com/amcz/hysplit_metdata
charleshelms
Posts: 2
Joined: December 9th, 2020, 12:20 pm
Registered HYSPLIT User: Yes

Re: ERA5 dataset

Post by charleshelms »

Sorry for the delayed response (I didn't notice that the forum thread went onto a second page :? )

I managed to get data that era52arl found agreeable based on the examples in the github readme file. That said, I suspect the cause of my issues might actually be down to me using 'cat' to combine the single-time grib files I downloaded into one combined grib file expected by era52arl instead of using grib_copy (I could have sworn 'cat' was a valid way to combine grib files, but perhaps it's a case of being good enough for some uses but not for all, or perhaps I'm just misremembering). As far as specifying 'gridType' as 'regular_ll', I wasn't able to get that to work, but after digging around it might be that I should have specified 'typeOfGrid', although I haven't had a chance to test that. Either way, I've included examples of the python scripts that I used to download the individual files that I then combined with grib_copy into three files (one each for the pressure levels, the surface analysis, and the surface forecast). One final note, I haven't had the chance to run the trajectory model on the converted files (and I probably won't have time to do so for a couple weeks), but I wanted to reply to the thread in a semi-timely manner. I did notice that I kept getting the following warning:
"WARNING, 2d forecast does not have time 13 9 5 9"
but I don't expect it to be an issue as the last time I'm interested in is 0000 UTC 5 September 2013.

Thanks again for your help!

Here are the examples of the python scripts I used:

::::::::::::::
tmpera5grab_pl_an.py
::::::::::::::
#!/PATH/TO/anaconda3/bin/python
import cdsapi
c = cdsapi.Client()
c.retrieve('reanalysis-era5-pressure-levels',
{
'variable' : ['temperature','u_component_of_wind','v_component_of_wind','vertical_velocity','relative_humidity','geopotential'],
'pressure_level':['10','20','30','50','70','100','125','150','175','200','225','250','300','350','400','450','500','550','600','650','700','750','775','800','825','850','875','900','925','950','975','1000'],
'product_type' : 'reanalysis',
'grid' : '0.25/0.25',
'date':'2013-09-05',
'time': '00:00:00',
'type': 'an',
'area': [50, -105, 0, -10],
'format':'grib'
}, 'era5_atlantic_201309050000')
::::::::::::::
tmpera5grab_sfc_an.py
::::::::::::::
#!/PATH/TO/anaconda3/bin/python
import cdsapi
c = cdsapi.Client()
c.retrieve('reanalysis-era5-single-levels', {
'variable' : ['2m_temperature','10m_u_component_of_wind','10m_v_component_of_wind','surface_pressure','boundary_layer_height','geopotential','friction_velocity'],
'product_type' : 'reanalysis',
'grid' : '0.25/0.25',
'type': 'an',
'date':'2013-09-05',
'time': '00:00:00',
'area': [50, -105, 0, -10],
'format':'grib'
}, 'era5_sfc_an_atlantic_201309050000')
::::::::::::::
tmpera5grab_sfc_fc.py
::::::::::::::
#!/PATH/TO/anaconda3/bin/python
import cdsapi
c = cdsapi.Client()
c.retrieve('reanalysis-era5-single-levels', {
'variable' : ['2m_temperature','10m_u_component_of_wind','10m_v_component_of_wind','surface_pressure','boundary_layer_height','geopotential','friction_velocity'],
'product_type' : 'reanalysis',
'grid' : '0.25/0.25',
'type': 'fc',
'date':'2013-09-04',
'time': '18:00:00',
'area': [50, -105, 0, -10],
'format':'grib'
},
'era5_sfc_fc_atlantic_201309041800')
alicec
Posts: 260
Joined: February 8th, 2016, 12:56 pm
Registered HYSPLIT User: Yes

Re: ERA5 dataset

Post by alicec »

Thanks for updating and sharing your scripts.

A few thoughts.

With the new cdsapi you do not need to have separate files for the analysis and forecast surface data.
The new api allows you to retrieve both types of variables into the same file (The old api did not).
The era52arl program can still ingest separate files, but you can also just give it one file for the surface fields.
The -f option is optional.
I strongly suggest you use a single file for the surface fields.

The warning you are getting relates to the use of the separate forecast file.
With the old api, the forecast data had to be retrieved separately and then the conversion program needed to calculate the
date the data was valid for by taking into account the forecast step. The new api automatically packages the forecast
data in with the analysis data so this does not need to be done.

specifying
'grid':'0.25/0.25'
results in the regular_ll grid type.
https://confluence.ecmwf.int/display/CK ... +reference
Post Reply