Multiprocessor run using particle input file (PARINIT)

Post any questions or comments regarding the LINUX version of HYSPLIT. This includes the model execution, GUIs, results, or graphics. Be sure to mention the version of LINUX you are using.
kdgeorge
Posts: 8
Joined: May 20th, 2013, 12:26 pm
Registered HYSPLIT User: Yes

Multiprocessor run using particle input file (PARINIT)

Post by kdgeorge » February 20th, 2014, 4:33 pm

I am having some problems running a multiprocessor volcanic ash run while also reading a particle input file (PARINIT). I am using the command:

../exec/run_mpi.sh 4 hysm_std

The simulation *appears* to run correctly, but after creating an image from the output it is obvious that there were no particles read from the input file. I assumed there might be a problem with reading one PARINIT file from a previous run. Therefore, I attempted to rerun the previous cycle and copy the multiple PARDUMP.001 - .004 files to PARINIT.001 - .004 files and rerun the new simulation with these input files. The simulation again *seemed* to run correctly, but did not produce a product with ingested particles.

Does the multiprocessor option not work with one or multiple PARINIT files? Am I missing some other option that should be turned on in the SETUP file?

Thanks for any help!

ariel.stein
Posts: 629
Joined: November 7th, 2012, 3:14 pm
Registered HYSPLIT User: Yes
Contact:

Re: Multiprocessor run using particle input file (PARINIT)

Post by ariel.stein » February 20th, 2014, 5:08 pm

You need to set your input and output PARDUMP files in the SETUP.CFG
pinpf = 'PARINI',
poutf = 'PAROUT',

kdgeorge
Posts: 8
Joined: May 20th, 2013, 12:26 pm
Registered HYSPLIT User: Yes

Re: Multiprocessor run using particle input file (PARINIT)

Post by kdgeorge » February 20th, 2014, 5:10 pm

Yes, the correct input and output files are set in the SETUP file as:

pinpf = 'PARINIT',
poutf = 'PARDUMP',

ariel.stein
Posts: 629
Joined: November 7th, 2012, 3:14 pm
Registered HYSPLIT User: Yes
Contact:

Re: Multiprocessor run using particle input file (PARINIT)

Post by ariel.stein » February 20th, 2014, 5:14 pm

Are you also setting :
ndump = 1,
ncycl = 1,

or
ndump = 24,

on the SETUP.CFG?

ariel.stein
Posts: 629
Joined: November 7th, 2012, 3:14 pm
Registered HYSPLIT User: Yes
Contact:

Re: Multiprocessor run using particle input file (PARINIT)

Post by ariel.stein » February 21st, 2014, 10:42 am

More information:

Particle File Initialization Options

NINIT sets the type of initialization performed. There are two types of initializations. One occurs at model startup, prior to the start of the calculations. This sets the initial conditions for the calculation, any particles on the computational grid at model startup are loaded. In the second situation, during the model simulation, particles are loaded each time step, if a matching time is found in the boundary condition file. These particles may represent emissions from a previous calculation and can be added to or replace the existing particles in the simulation. When NINIT is set to "0" no particle initialization occurs even if the input file is defined. A value of "1" reads the file only during the initialization process. No initialization will occur if the time of the particle dump does not match the time of the model initialization. A value of "2" will check the file each hour during the simulation, and if the times match, the particles will be added to those already contained in the simulation. A value of "3" is similar to the previous case, except the particles in the file replace all the particles in the simulation.

PINPF sets the default name PARINIT for the particle input file that can be used for initialization or boundary conditions. Note that particle files are just a dump of all the pollutants tracked by the model at a particular time and the file can consist of either puffs, particles, or a combination of both.

PINBC with a default name PARINBC is a special file that can be used for boundary conditions while PINPF can be used for initial conditions or boundary conditions. Setting the PINBC file name is not available through the GUI. For instance, PINPF can be used for both initial (NINIT=1) and boundary conditions (NINIT>1), but when PINBC is also defined, then it would replace PINPF after the initial input time.

Also not yet available through the GUI, the namelist variable KHINP<>0 sets the particle age that will be read from the particle initialization file that may contain particles of many different ages each output time period. This option is intended to be used with continuous initialization (NINIT=2). An example of this application may be to create a particle file using high resolution regional meteorology, but only for the first few hours of transport. In the case of continuous emissions, this file would be read each time period during the coarser grid simulation, but only initializing with particles that are of age = KHINP. The final output requires that the concentration grids from the two simulations be added together.

Also note that when a particle file is used for initialization of a simulation, the meteorological grid identification of a particle is retained from the previous simulation (except when KHINP<>0). This means that if different meteorological input files are defined in the two simulations, a particle on the second simulation may not be defined on the intended meteorological grid at the start of the calculation.

Particle File Output Options

NDUMP can be set to dump out all the particle/puff points at selected time intervals to a file called PARDUMP. This file can be read from the root directory at the start of a new simulation to continue the previous calculation. Valid NDUMP settings are [0] for no I/O or [hours] to set the number of hours from the start of the simulation at which all the endpoint positions will be written to the file. A negative value for NDUMP will dump out the particle/puff points at each model time step from the start of the simulation up to NDUMP hours. The file must exist in the root directory and NDUMP<>0 for the model to initialize pollutant particles from the file. NDUMP is used in conjunction with NCYCL (see below).

NCYCL sets the repeat interval at which the PARDUMP file is to be written after the first write at hours NDUMP. For instance, in a multi-day simulation, one application would be to set NDUMP=24 and NCYCL=24 to output all points at the end of every simulation day. If the model were to crash unexpectedly, the simulation could be restarted from the last PARDUMP output. If NCYCL is less than zero, then the PARDUMP file is over-written, rather than appended, at that interval. This means that the file will always contain one time period, a more compact file for initialization purposes. Also note that if NDUMP is negative, the NCYCL field is ignored.

POUTF is default name for the particle dump output file PARDUMP.

kdgeorge
Posts: 8
Joined: May 20th, 2013, 12:26 pm
Registered HYSPLIT User: Yes

Re: Multiprocessor run using particle input file (PARINIT)

Post by kdgeorge » February 21st, 2014, 4:25 pm

I have set it to output every hour because of how I create my images.

ndump = 1,
ncycl = 1,

I can get it to work fine using one processor (hycs_std) but the multiprocessor run (hycm_std) fails to ingest the PARINIT file.

ariel.stein
Posts: 629
Joined: November 7th, 2012, 3:14 pm
Registered HYSPLIT User: Yes
Contact:

Re: Multiprocessor run using particle input file (PARINIT)

Post by ariel.stein » February 24th, 2014, 8:04 am

Are you renaming the files accordingly?
Also , please download the latests version of the code.

kdgeorge
Posts: 8
Joined: May 20th, 2013, 12:26 pm
Registered HYSPLIT User: Yes

Re: Multiprocessor run using particle input file (PARINIT)

Post by kdgeorge » February 24th, 2014, 12:42 pm

I'm not sure exactly what you mean by renaming the files.

My steps to run a simulation include running the first multiprocessor run (say for 6 hours with 06Z GFS and 4 processors) and getting 4 output files (PARDUMP.001, PARDUMP.002, PARDUMP.003, and PARDUMP.004). These I usually combine with the parmerge program to make my graphics and rename the single PARDUMP file to something useful.

To make the next run (say for 6 hours with 12Z GFS data) I assume I would have to rename the output from the 6Z run to PARINIT.001, PARINIT.002, PARINIT.003, PARINIT.004. Then I would run my new multiprocessor run and get a new set of 4 PARDUMP files (one from each processor). There are no errors thrown when I make this run, but it is obvious from the output that the PARINIT files were not read at the initialization. If I were to do this same run, but with one PARINIT file, the output looks correct.

An example SETUP file would be:

&SETUP
delt = 2,
tratio = 0.75,
initd = 0,
kpuff = 0,
khmax = 120,
numpar = 18000,
qcycle = 0.0,
efile = '',
tkerd = 0.18,
tkern = 0.18,
ninit = 1,
ndump = 1,
ncycl = 1,
pinpf = 'PARINIT',
poutf = 'PARDUMP',
mgmin = 10,
kmsl = 1,
maxpar = 300000,
cpack = 1,
cmass = 0,
dxf = 1.0,
dyf = 1.0,
dzf = 0.01,
ichem = 0,
maxdim = 1,

I usually download the latest version every year and work through to get it modified for my systems; however, I have not done it yet this year because it takes a little bit of work to modify the some of the hard coded settings. I believe the last version I have is #480.

ariel.stein
Posts: 629
Joined: November 7th, 2012, 3:14 pm
Registered HYSPLIT User: Yes
Contact:

Re: Multiprocessor run using particle input file (PARINIT)

Post by ariel.stein » February 24th, 2014, 1:24 pm

Can you please send me any of the MESSAGE files you are getting when running the second cycle?

kdgeorge
Posts: 8
Joined: May 20th, 2013, 12:26 pm
Registered HYSPLIT User: Yes

Re: Multiprocessor run using particle input file (PARINIT)

Post by kdgeorge » February 24th, 2014, 4:32 pm

The MESSAGE.001 file begins with the below text.

I notice from looking at this that the PARDUMP file is designated by .001 (and .002, .003, .004 in the others), but the PARINIT file is not. So, it makes be believe they are not being read in properly by each processor. However, I did try the same run with only one PARINIT file used across all processors with a similar incorrect result (which I expected). I wasn't sure what the warning about parinp was referring to at the bottom of the MESSAGE file.


Simulation Date (CCYYMMDD): 20140220
Simulation Time (HHMMSS.S): 200847.768
NOTICE main: number meteo grids and times - 1 1

HYSPLIT4
USE: svn info | tail -n3
to fill in these three records
with the version information!
------------- Start Namelist configuration -------------------
Internal grid parameters (nlvl,aa,bb,cc): 34 30.00000 -25.00000 5.000000
&SETUP
INITD= 0,
KHMAX= 120,
NUMPAR= 18000,
MAXPAR= 300000,
MAXDIM= 1,
QCYCLE= 0.000000 ,
P10F= 1.000000 ,
KMIXD= 0,
KMIX0= 250,
FRME= 0.1000000 ,
FRMR= 0.000000 ,
KRND= 6,
DELT= 2.000000 ,
KDEF= 0,
TKERD= 0.1800000 ,
TKERN= 0.1800000 ,
NDUMP= 1,
NCYCL= 1,
TRATIO= 0.7500000 ,
ISOT= -99,
MGMIN= 10,
KMSL= 1,
NSTR= 0,
MHRS= 0,
NVER= 0,
CPACK= 1,
CMASS= 0,
ICHEM= 0,
EFILE= ,
VSCALE= 200.0000 ,
VSCALES= 200.0000 ,
VSCALEU= 200.0000 ,
K10M= 1,
VDIST=VMSDIST ,
TOUT= 0,
TM_PRES= 0,
TM_TPOT= 0,
TM_TAMB= 0,
TM_RAIN= 0,
TM_MIXD= 0,
TM_TERR= 0,
HSCALE= 10800.00 ,
TM_DSWF= 0,
TM_RELH= 0,
TM_SPHU= 0,
TM_MIXR= 0,
DXF= 1.000000 ,
DYF= 1.000000 ,
DZF= 9.9999998E-03,
PINPF=PARINIT ,
POUTF=PARDUMP.001 ,
NINIT= 1,
KAGL= 0,
KPUFF= 0,
KBLT= 2,
KZMIX= 0,
TVMIX= 1.000000 ,
KBLS= 1,
CONAGE= 48,
KSPL= 1,
FRHS= 1.000000 ,
FRVS= 9.9999998E-03,
FRTS= 0.1000000 ,
FRHMAX= 3.000000 ,
SPLITF= 1.000000 ,
MESSG=MESSAGE ,
NBPTYP= 1,
RHB= 80,
RHT= 60,
OUTDT= 0,
DECAY= 1,
KWET= 1,
KRAND= 1, /

------------- End Namelist configuration -------------------

NOTICE main: pollutant initialization flags
Wet deposition - T
Grav settling - T
NOTICE main: meteorological data flags
Ten meter wind - T
Two meter temp - T
Exchange coeff - F
Heat flux - T
Momentum flux - T
Velocity variance - F
3D TKE field - F
Down shortwave - T
Friction velocity - F
Friction temp - F
Terrain height - T
Surface pressure - T
Mixed layer depth - T
Profile averaging - 0
Stability method - 1
Horizontal mixing - 0
PBL mixing scheme - 2
Free Trop Mixing - 1.000000
Precip accumulate - 360
------------- Start computation messages -------------------
WARNING parinp: initialization in time wait mode
NOTICE metpos: (mtime,ftime) - 60031440 0
NOTICE metpos: (mtime,ftime) - 60031620 0
NOTICE advpnt: (kg,nx,ny,nz) - 1 10 10 34
NOTICE sfcinp: reading ASCDATA.CFG
NOTICE sfcinp: record terminator = CR/LF
NOTICE metgrd: (kg, xyr,xy1) - 1 10 10 109 79

Post Reply