Problem setting up cluster analysis

Topics about the trajectory clustering program for HYSPLIT.
farai
Posts: 6
Joined: May 11th, 2020, 5:40 pm
Registered HYSPLIT User: No

Problem setting up cluster analysis

Post by farai » July 20th, 2020, 3:09 pm

I am trying to run a 24 hr cluster analysis on 35 backward trajectories. I have the placed the tdump files in the endpts folder and am able to generate the INFILE without any problem. I have manually inspected the INFILE and everything looks good (has all the files I need). However, when I run the cluster analysis I get the following prompt also shown in attachment:

Model started ... it is slower at the beginning
Started Cluster (Version: April 2015)
Number of trajectories in INFILE: 35
Pass 1 out of 1
Complete Cluster - number of trajectories clustered:

The TSV is empty. If I ignore this and try lookup the final number of clusters I get an error ehich I have expanded the details below:

Started Clusend (Version: November 2009)
min # trajectories, 15 , not met ( 3 )
504
Started Clusend (Version: November 2009)
min # trajectories, 15 , not met ( 3 )
504
while executing
"exec $exec_dir/clusend.exe $arg1 $arg2 $arg3 $arg4 $arg5 $arg6"
(procedure "List_outcomes" line 43)
invoked from within
"List_outcomes"
invoked from within
".trajclus.ncout.cont invoke"
("uplevel" body line 1)
invoked from within
"uplevel #0 [list $w invoke]"
(procedure "tk::ButtonUp" line 24)
invoked from within
"tk::ButtonUp .trajclus.ncout.cont"
(command bound to event)

Since I have 35 trajectories I don't get how I failing to meet the minimum requirements. I appreciate any feedback because I can't figure out where I am going wrong or what the issue is. I am also using the GUI.
Attachments
error.PNG
error.PNG (8.18 KiB) Viewed 250 times
assign_trajectories.PNG
assign_trajectories.PNG (8.51 KiB) Viewed 250 times
hysplit_cluster.PNG
hysplit_cluster.PNG (10.5 KiB) Viewed 250 times

barbara.stunder
Posts: 416
Joined: November 9th, 2012, 4:23 pm
Registered HYSPLIT User: Yes

Re: Problem setting up cluster analysis

Post by barbara.stunder » July 21st, 2020, 7:58 am

Open the text files TCLUS and TNOCLUS. TCLUS lists the trajectories clustered. TNOCLUS lists the trajectories not clustered, usually because of the 'Hours to cluster' and "Trajectory skip" inputs. If all the trajectories were used, TCLUS would be the same as INFILE. Because it says "Pass 1 out of 1", that is implying only 1 trajectory was used.

Do you have Hours to cluster 24? Are all the trajectory durations at least 24-hours?
Do you have Trajectory skip "1"? That means use every trajectory. If it were 2, it would be using every 2nd trajectory (every other).

farai
Posts: 6
Joined: May 11th, 2020, 5:40 pm
Registered HYSPLIT User: No

Re: Problem setting up cluster analysis

Post by farai » July 21st, 2020, 12:57 pm

TCLUS contains two files
TNOCLUS contains 33files

All my files contain 120hrs of trajectory and I have tried 18, 24, 36 hrs to check if it may be related to the hours to cluster but it seems to make no difference.
I have also been using Trajectory skip "1" as I need all the trajectories, given that I only have 35.

barbara.stunder
Posts: 416
Joined: November 9th, 2012, 4:23 pm
Registered HYSPLIT User: Yes

Re: Problem setting up cluster analysis

Post by barbara.stunder » July 21st, 2020, 4:11 pm

Try to attach the files here so I can run a test.

farai
Posts: 6
Joined: May 11th, 2020, 5:40 pm
Registered HYSPLIT User: No

Re: Problem setting up cluster analysis

Post by farai » July 21st, 2020, 4:45 pm

I'm assuming you want the tdump files? If so, I can't load them. Am getting an invalid file type error. Is there a work around on this.

Thanks

barbara.stunder
Posts: 416
Joined: November 9th, 2012, 4:23 pm
Registered HYSPLIT User: Yes

Re: Problem setting up cluster analysis

Post by barbara.stunder » July 22nd, 2020, 10:13 am

Please just email the tdump files to me at barbara.stunder at noaa dot gov (but with @ and .)

farai
Posts: 6
Joined: May 11th, 2020, 5:40 pm
Registered HYSPLIT User: No

Re: Problem setting up cluster analysis

Post by farai » July 22nd, 2020, 11:51 am

Just sent them through. Thanks

barbara.stunder
Posts: 416
Joined: November 9th, 2012, 4:23 pm
Registered HYSPLIT User: Yes

Re: Problem setting up cluster analysis

Post by barbara.stunder » July 23rd, 2020, 9:02 am

I reproduced the issue. Need a little time to figure it out.

barbara.stunder
Posts: 416
Joined: November 9th, 2012, 4:23 pm
Registered HYSPLIT User: Yes

Re: Problem setting up cluster analysis

Post by barbara.stunder » July 23rd, 2020, 1:42 pm

The two trajectories that were listed in the TCLUS file (used in the clustering) started at 59 minutes after the hour.
The other trajectories, per spot-checking, started at other non-zero minutes, such as 14, 29, etc.

Rerun the trajectories so they always start at 59 minutes (or 00 minutes) after the hour, then the clustering should work.

We need to revise the cluster documentation to state the trajectories need to start on the hour, and add a relevant output message in the cluster program, or generalize the code to handle minutes.

farai
Posts: 6
Joined: May 11th, 2020, 5:40 pm
Registered HYSPLIT User: No

Re: Problem setting up cluster analysis

Post by farai » July 23rd, 2020, 2:26 pm

Thanks so much. I will rerun this and give feedback

Post Reply