Linux .dbf table output inconcistent with like-for-like Windows output

[mwj202]

Hi All,

I've recently run HYSPLIT for Linux using pretty much the same .tcl script that I used for model runs on Windows but I'm having an issue with the shapefiles outputs. These couldn't be loaded into ArcGIS (following ftp transfer to Windows) because there are some problems with field types, and on closer inspection I have found an issue with the output .dbf tables that accompany the shapefiles. As you can see from the .dbf file info below (viewed in linux using python's dbfview package), where the "Trajnum" field should be populated with a 4-digit integer (1000, 1001, ..) I have 6-character data like 1000 02, 1001 02, 1001 03 which I presume is registered as a string field, hence the field type error when trying to open in ArcGIS. I also have an additional field "1000 201" which clearly just shouldn't be there.

Code: Select all

Trajnum    : 1001 2
Yyyymmdd   : 0111231 2
Time       : 300
Level      : 0.
   1000 201:

Trajnum    : 1002 2
Yyyymmdd   : 0111231 2
Time       : 200
Level      : 0.
   1000 201:
   

I'm at a loose end as to why the outputs from the same tcl commands would produce a different output in linux vs. windows, however, I'm pretty sure that the problem must be occurring in the following code sequence (NB appears in a look, hence all the $):

Code: Select all

exec $exeDir/trajplot -i$EPcodex$YY$MM$DD -a1 -l1 -o$EPcodex$YY$MM$DD
exec $exeDir/ascii2shp -i p$EPcodex$YY$MM$DD points <GIS_traj_ps_01.txt
exec $exeDir/txt2dbf -C7 -C9 -C5 -C9 -d, -d, -d, GIS_traj_ps_01.att p$EPcodex$YY$MM$DD.dbf

In particular I think the options that I've used for txt2dbf "-C7 -C9 -C5 -C9 -d, -d, -d," are likely to be the problem.. but I'm unsure about what options to use or how to figure this out. Does anyone know what each option in that list means? Or could anyone point me towards a good resource?

Any pointers very much appreciated.
Matt

[mwj202]

Update - I've been playing around with the txt2dbf options "-C7 -C9 -C5 -C9 -d, -d, -d," and I've found that if I change to "-C8 -C9 -C5 -C9 -d, -d, -d," I get closer to what I want, BUT there's still that pesky extra field that's unexplained and the first record isn't being recorded in the .dbf.

Code: Select all

Trajnum    : 1001
Yyyymmdd   : 20111231
Time       : 2300
Level      : 9.
   1000  20:

It seems that the first x characters of the first record are being treated as a field, i.e. "1000 2" (when using -C7 option) or "1000 20" (when using -C8 option) and then record 2 (1001) is being treated as the first one to be converted. Perhaps some problem with delimiting the last field (NB ends with a "." even if I change to an integer by replacing -C9 with -I9)..?

[glenn.rolph]

txt2dbf will convert the ASCII Generate formatted data into shapefiles. Look in your .att file that was created with ascii2shp and see if it looks similar at this:

#TRAJNUM,YYYYMMDD,TIME,LEVEL
1000,19951016,0000, 991.
1001,19951016,0100, 991.
1002,19951016,0200, 991.
1003,19951016,0300, 992.
1004,19951016,0400, 994.
1005,19951016,0500, 994.
1006,19951016,0600, 995.
1007,19951016,0700, 996.
1008,19951016,0800, 996.
1009,19951016,0900, 997.
1010,19951016,1000, 998.
1011,19951016,1100, 999.
1012,19951016,1200, 1001.
2000,19951016,0000, 934.
2001,19951016,0100, 936.
2002,19951016,0200, 937.
2003,19951016,0300, 940.
2004,19951016,0400, 943.
2005,19951016,0500, 946.
2006,19951016,0600, 951.
2007,19951016,0700, 954.
2008,19951016,0800, 957.
2009,19951016,0900, 960.
2010,19951016,1000, 963.
2011,19951016,1100, 964.
2012,19951016,1200, 964.
3000,19951016,0000, 879.
3001,19951016,0100, 882.
3002,19951016,0200, 882.
3003,19951016,0300, 883.
3004,19951016,0400, 886.
3005,19951016,0500, 891.
3006,19951016,0600, 899.
3007,19951016,0700, 907.
3008,19951016,0800, 908.
3009,19951016,0900, 908.
3010,19951016,1000, 909.
3011,19951016,1100, 909.
3012,19951016,1200, 905.

Its possible that you have a space in the first number and that may be causing problems with txt2dbf.

There is also a help file in the /hysplit/trunk/gisprog/dbf directory that may be of some use in understanding the options for txt2dbf or where to find more information on that 3rd party C program.

[mwj202]

Thanks glenn.rolph,

The .att table is laid out as follows:

Code: Select all

#TRAJNUM,YYYYMMDD,TIME,LEVEL
  1000,20120101,0000,      10.
  1001,20111231,2300,       9.
  1002,20111231,2200,       9.
  1003,20111231,2100,      10.
  1004,20111231,2000,      12.
 

There are two spaces before the first field is shown e.g. <space><space>1000. Is this likely to be the problem? And how might this be addressed?

Thanks for pointing me towards the txt2dbf

[mwj202]

For the benefit of other users, progress on this issue (and the ultimate solution) can be found here:

http://stackoverflow.com/questions/4214 ... bf-package

[glenn.rolph]

If you find that we have an error in our txt or att files that are causing your problem, let us know what is wrong and we will attempt to correct it.