Aggregating many simultaneous concentration files
Posted: February 27th, 2023, 5:27 pm
I am exploring the idea of using HYSPLIT to simulate many active simultaneous wildfires. To do this, I have run each wildfire event individually (with each event being run in parallel over a distributed cluster). The result is many concentration files which overlap in both time and space. I need to be able to combine these files into a single gridded set of concentration time series.
I have attempted to use conmerge to do this, however I have not been able to get it to work -- perhaps because there are too many files? It has worked for smaller subsets of the output files, but not the full batch. Instead, I have converted my output files to netCDF at the end of each parallel process so that I can attempt to combine them manually.
My question: what is the correct way to sum concentration files? Is it a literal sum of concentration along the process dimension of my array? When I try this, I get very very large concentration values and I'm concerned that the correct approach might not be a simple linear sum.
It may also simply be the settings I have used -- having taken the wildfire example as a template.
I have attempted to use conmerge to do this, however I have not been able to get it to work -- perhaps because there are too many files? It has worked for smaller subsets of the output files, but not the full batch. Instead, I have converted my output files to netCDF at the end of each parallel process so that I can attempt to combine them manually.
My question: what is the correct way to sum concentration files? Is it a literal sum of concentration along the process dimension of my array? When I try this, I get very very large concentration values and I'm concerned that the correct approach might not be a simple linear sum.
It may also simply be the settings I have used -- having taken the wildfire example as a template.