Cluster Analysis not finishing

[egribbin]

I have a collection of approximately 1400 trajectories I am trying to cluster for an ozone site in San Antonio. I have tried to cluster all 1400 and just every other one (i.e. 699 trajectories).

Computer details:
Dell Optiplex with 10 GB RAM (Supplemented by a 16GB Flash Drive on Ready Boost)
Windows 7 Professional 64-bit OS

My Cluster setup:

Cluster Setup

When I run the analysis I have the following simulation log displayed:

Simulation Log

simulationlog.png (10.84 KiB) Viewed 10798 times

and I get the following error message:

Error Message

ErrorMessage.png (11.66 KiB) Viewed 10798 times

I'm not sure what is causing this error and this amount of memory is usually enough to run the analysis for 700 48-hour trajectories.
Even more mysterious is the fact that while the simulation log indicates that I am only on the first "Pass", the CLUSTER file in the working directory indicates that the analysis has actually performed 180 "Passes".
Can anyone show me the error of my ways? Many thanks in advance!!!

Erik Gribbin

[barbara.stunder]

Run cluster.exe "manually" from the command prompt to see if you get an error message that is not shown in the Simulation Log window.

Start - All Programs - Accessories - Command Prompt
cd <the cluster working directory>
..\..\exec\cluster.exe

If the file CCONTROL exists in the working directory, which it probably does from your last run, it will read those inputs. If that file doesn't exist, you will be prompted to add the number of hours to cluster, etc.

[egribbin]

As a test I used the GUI to enter all of the set up information and make the infile using a subset of the trajectories I'm working with (166 trajectories). In the GUI, I set the working directory as C:/Users/egribbin/DeskTop/Bullis_Test/Working/ and the endpoints folder as C:/Users/egribbin/DeskTop/Bullis_Test/. When I went to the command line and ran cluster.exe there was no ccontrol file so I had to enter some information manually. After that, cluster.exe could not find the infile which was stored in the "working" directory and execution stopped.

Command Line History

Just for grins, I hit the "Run Cluster Analysis" button, and HYSPLIT had no trouble finding the infile and the simulation log went through all 165 passes without generating an error message. The Total Spatial Variance displayed thus:

Total Spatial Variance

clusplot.png (2.5 KiB) Viewed 10792 times

...and I chose 7 clusters. The cluster means plotted like this:

Cluster Means 1-7

clusmean1_7.png (10.24 KiB) Viewed 10792 times

...Which I'm going to assume is not a "good" outcome. Should I be keeping the working directory as c:\hysplit4\exec\?

[egribbin]

I should add that I deleted all files in the "working" from my previous attempts before running the test you suggested.

E

[barbara.stunder]

Copy INFILE to C:/Users/egribbin/DeskTop/Bullis_Test/
Re-run the c:\hysplit4\exec\cluster.exe. You should get the same results.
How you set the directories in the GUI has no bearing on the run in the Command prompt window.

However, as you say, from the cluster means plot, it doesn't look right. Are you able zip those endpoints files and put them on an ftp server so I can get them or email them to me so I can do a run here? Barbara.Stunder@noaa.gov

[grossiclaud]

dear Barbara,
I have tried to plot the cluster means with 8 cluster but my plot looks like the previous one shown here.
Any suggestions?

Thanks
Claudia

[grossiclaud]

here you can see the result I got for 7 cluster analysis.
I do not understand which could be the problem becasue i changed the all possible options.

Claudia

[sandygrad]

I am having (what appears to be) this exact same problem as "grossiclaud". I am running 72 hour back trajectories and trying to run a cluster analysis on them, but the cluster analysis seems to only be analyzing the first point, not the whole trajectory. How can I get it to analyze the whole trajectory? Is there a setting I am missing or do I need to alter the format of my trajectories?

[barbara.stunder]

I haven't found the particular issue with this, however try these:
Re-install the current version of HYSPLIT and re-run.
If you have a 64-bit PC, try both the 32-bit and 64-bit versions.

If it is still not working, can you send me the trajectories or let me ftp them so I can look at this.

[sandygrad]

Thanks barbara.stunder

Although I have 64bit PC, installing the 32bit version did the trick, and I was able to run the cluster analysis on my full 72-hour trajectories.