Page 1 of 2

Problem setting up cluster analysis

Posted: July 20th, 2020, 3:09 pm
by farai
I am trying to run a 24 hr cluster analysis on 35 backward trajectories. I have the placed the tdump files in the endpts folder and am able to generate the INFILE without any problem. I have manually inspected the INFILE and everything looks good (has all the files I need). However, when I run the cluster analysis I get the following prompt also shown in attachment:

Model started ... it is slower at the beginning
Started Cluster (Version: April 2015)
Number of trajectories in INFILE: 35
Pass 1 out of 1
Complete Cluster - number of trajectories clustered:

The TSV is empty. If I ignore this and try lookup the final number of clusters I get an error ehich I have expanded the details below:

Started Clusend (Version: November 2009)
min # trajectories, 15 , not met ( 3 )
504
Started Clusend (Version: November 2009)
min # trajectories, 15 , not met ( 3 )
504
while executing
"exec $exec_dir/clusend.exe $arg1 $arg2 $arg3 $arg4 $arg5 $arg6"
(procedure "List_outcomes" line 43)
invoked from within
"List_outcomes"
invoked from within
".trajclus.ncout.cont invoke"
("uplevel" body line 1)
invoked from within
"uplevel #0 [list $w invoke]"
(procedure "tk::ButtonUp" line 24)
invoked from within
"tk::ButtonUp .trajclus.ncout.cont"
(command bound to event)

Since I have 35 trajectories I don't get how I failing to meet the minimum requirements. I appreciate any feedback because I can't figure out where I am going wrong or what the issue is. I am also using the GUI.

Re: Problem setting up cluster analysis

Posted: July 21st, 2020, 7:58 am
by barbara.stunder
Open the text files TCLUS and TNOCLUS. TCLUS lists the trajectories clustered. TNOCLUS lists the trajectories not clustered, usually because of the 'Hours to cluster' and "Trajectory skip" inputs. If all the trajectories were used, TCLUS would be the same as INFILE. Because it says "Pass 1 out of 1", that is implying only 1 trajectory was used.

Do you have Hours to cluster 24? Are all the trajectory durations at least 24-hours?
Do you have Trajectory skip "1"? That means use every trajectory. If it were 2, it would be using every 2nd trajectory (every other).

Re: Problem setting up cluster analysis

Posted: July 21st, 2020, 12:57 pm
by farai
TCLUS contains two files
TNOCLUS contains 33files

All my files contain 120hrs of trajectory and I have tried 18, 24, 36 hrs to check if it may be related to the hours to cluster but it seems to make no difference.
I have also been using Trajectory skip "1" as I need all the trajectories, given that I only have 35.

Re: Problem setting up cluster analysis

Posted: July 21st, 2020, 4:11 pm
by barbara.stunder
Try to attach the files here so I can run a test.

Re: Problem setting up cluster analysis

Posted: July 21st, 2020, 4:45 pm
by farai
I'm assuming you want the tdump files? If so, I can't load them. Am getting an invalid file type error. Is there a work around on this.

Thanks

Re: Problem setting up cluster analysis

Posted: July 22nd, 2020, 10:13 am
by barbara.stunder
Please just email the tdump files to me at barbara.stunder at noaa dot gov (but with @ and .)

Re: Problem setting up cluster analysis

Posted: July 22nd, 2020, 11:51 am
by farai
Just sent them through. Thanks

Re: Problem setting up cluster analysis

Posted: July 23rd, 2020, 9:02 am
by barbara.stunder
I reproduced the issue. Need a little time to figure it out.

Re: Problem setting up cluster analysis

Posted: July 23rd, 2020, 1:42 pm
by barbara.stunder
The two trajectories that were listed in the TCLUS file (used in the clustering) started at 59 minutes after the hour.
The other trajectories, per spot-checking, started at other non-zero minutes, such as 14, 29, etc.

Rerun the trajectories so they always start at 59 minutes (or 00 minutes) after the hour, then the clustering should work.

We need to revise the cluster documentation to state the trajectories need to start on the hour, and add a relevant output message in the cluster program, or generalize the code to handle minutes.

Re: Problem setting up cluster analysis

Posted: July 23rd, 2020, 2:26 pm
by farai
Thanks so much. I will rerun this and give feedback