meteorology data stored in several files

zhy566
Posts: 14
Joined: July 2nd, 2017, 11:30 pm
Registered HYSPLIT User: No

meteorology data stored in several files

Post by zhy566 » October 3rd, 2017, 4:20 pm

Hi,

I am trying to convert the ERA-interim data into arl format. However, the variables are not stored in the same file.
For example, I have one file of u, v, w; and another file of T, q and z. The surface file includes 10m u and v, 2m T and dewpoint T as well as other variables like mean sea level pressure and sea surface pressure.

My questions are:
1. In the process of converting the data, is it possible to use several files for the 3D variables? If so, Could you please show me a script example of how to do it? I am running the model in Linux.
2. Does it matter if the data file contains some extra variables that are not needed in HYSPLIT?

Thank you very much!

Best,
Yang

alicec
Posts: 175
Joined: February 8th, 2016, 12:56 pm
Registered HYSPLIT User: Yes

Re: meteorology data stored in several files

Post by alicec » October 5th, 2017, 11:27 am

Please use the ecm2arl conversion program for converting ERA Interim data.
This program is designed to take up to 4 files.

It was designed for a pressure level file and two surface field files (one with forecast and one with analysis variables) as well as an invariant file.
However, I think it will work with using two pressure level files and one surface field file. For this program the extra variables should not matter.
Please try and let me know if it works. You can run ecm2arl to see the options.
It would be something like

ecm2arl -i(file with u, v and w) -s(file with t,q,z) -e(file with 2t, 10u, 10v, 2d, etc) -c(invariant file with geopotential)

You can get file with geopotential here:
http://apps.ecmwf.int/datasets/data/int ... invariant/

The time fields in your files should match eachother - it looks like they are probably all analysis so hopefully all retrieved with time of 00/06/12/18 and step of 00. Let me know how it goes.

zhy566
Posts: 14
Joined: July 2nd, 2017, 11:30 pm
Registered HYSPLIT User: No

Re: meteorology data stored in several files

Post by zhy566 » October 9th, 2017, 6:47 pm

Hi Alicec,

Thank you for your help!

There is a version problem with the gfortran in my server, so I couldn't run the executables in the hysplit package.
The admin then installed the grib2arl in the server so I tried that one to see how it goes.

Two issues appeared:

1. It seems that the grib2arl can't recognize the meteorology data as a second input.
The command I used is:
grib2arl -i(file with u, v and w) -s(file with t,q,z) -s(file with 2t, 10u, 10v, 2d, etc) -c(invariant file with geopotential)

It stopped and I got the message of "WARNING: moisture fields missing (SPHU or RELH)" and "ERROR: height field required with pressure levels","stop 1131", as I looked through the message, it showed that the software opened the second file (with t q z) but was not able to read and match the variables it needs.

2. The grib2arl was running if it reads the tqz file first.
The command with no error is:
grib2arl -i(file with t,q,z) -s(file with u, v and w) -s(file with 2t, 10u, 10v, 2d, etc) -c(invariant file with geopotential)

In this case, even the grib2arl was running, it did not read the uvw file so the wind filed is missing.
And the CFG_ARL file shows as follows:

Level 0: .00000 4 SHGT U10M V10M T02M
Level 1: 1000.0 3 HGTS TEMP SPHU
Level 2: 975.00 3 HGTS TEMP SPHU
Level 3: 950.00 3 HGTS TEMP SPHU

It shows that the girb2arl is not reading the uvw file.

From what I tried here, I think maybe the separate meteorology files do not work for grib2arl.
Do you have any idea on this?

Thank you!!

Yang

alicec
Posts: 175
Joined: February 8th, 2016, 12:56 pm
Registered HYSPLIT User: Yes

Re: meteorology data stored in several files

Post by alicec » October 10th, 2017, 9:02 am

You are correct, the grib2arl program will not take more than one pressure field file.
I think at this point your options are-

* get the ecm2arl program to compile.
* modify the grib2arl program to read multiple input files
* download the data from ECMWF again with all the pressure fields in one file.


Alice

zhy566
Posts: 14
Joined: July 2nd, 2017, 11:30 pm
Registered HYSPLIT User: No

Re: meteorology data stored in several files

Post by zhy566 » October 10th, 2017, 10:18 am

Hi Alice,

Thank you for your suggestions. I am working on the compilation of ecm2arl.
But meanwhile, could you please tell me how to modify the grib2arl program so that it can read multiple files?

Many thanks!

Best,
Yang

alicec
Posts: 175
Joined: February 8th, 2016, 12:56 pm
Registered HYSPLIT User: Yes

Re: meteorology data stored in several files

Post by alicec » October 10th, 2017, 12:00 pm

The code is commented and documented.
It utilizes the grib_api library from ECMWF. You can find documentation for that here
https://software.ecmwf.int/wiki/display ... Codes+Home
Note that the functions we use are the same in ecCodes as in gribapi so you can
use the ecCodes documentation.


Cheers,
Alice

zhy566
Posts: 14
Joined: July 2nd, 2017, 11:30 pm
Registered HYSPLIT User: No

Re: meteorology data stored in several files

Post by zhy566 » October 26th, 2017, 8:54 pm

Hi Alice,

I have tried the ecm2arl command as you suggested, however, it still doesn't work.
The situation is the same with grib2arl:
It stopped and I got the message of "WARNING: moisture fields missing (SPHU or RELH)" and "ERROR: height field required with pressure levels","stop 1131", as I looked through the message, it showed that the software opened the second file (with t q z) but was not able to read and match the variables it needs.
Could you please tell me how to modify the code of ecm2arl so that it can read multiple 3d files in one command?

Thank you very much!

Best,
Yang

zhy566
Posts: 14
Joined: July 2nd, 2017, 11:30 pm
Registered HYSPLIT User: No

Re: meteorology data stored in several files

Post by zhy566 » October 26th, 2017, 9:16 pm

Hi Alice,

I noticed that the problem is in the subroutine ANALYZE in both grib2arl and ecm2arl.
Even though the maximum input files in the ecm2arl is four, but it still misses the second 3D file.

Perhaps I should modify this subroutine? Could you give me some advice on this?

Many thanks!


Best,
Yang

alicec
Posts: 175
Joined: February 8th, 2016, 12:56 pm
Registered HYSPLIT User: Yes

Re: meteorology data stored in several files

Post by alicec » October 27th, 2017, 9:25 am

Is there a reason you cannot download new files from ECMWF with the pressure level data all in one file?
This would seem to be the easiest course.
Unfortunately we do not have the resources to modify the code for every possible configuration of the meteorological files.

Yes - go ahead and play around with the code.
Are you inputting the invariant file
with geopotential heights ? That is usually where the height field required error comes from.
It probably isn't finding the moisture fields in the first file and that is why you are getting the "moisture fields missing" error.
You probably have to disable this in the code so it doesn't check for moisture fields in the first file.

You can also look at modifying the api2arl program which is in the /data2arl/api2arl directory.
This program uses the ecmwf gribapi library (and will soon be updated to use eccodes).
It is a simpler program with less error catching (I.e. it won't care if moisture fields are missing)
and may be easier for you to modify to suit your needs.

zhy566
Posts: 14
Joined: July 2nd, 2017, 11:30 pm
Registered HYSPLIT User: No

Re: meteorology data stored in several files

Post by zhy566 » October 27th, 2017, 10:42 am

Hi Alice,

Thank you for your help. I consider re-downloading everything as my last option because I am planning to run a 38-year trajectory, which requires an enormous amount of data, re-downloading may be too time-consuming.

I will study the code more to see if I can get a better way on this problem.

Thank you again!

Best,
Yang

Post Reply