Validating the California Puff (CALPUFF) Modelling System Using an Industrial Area in Accra, Ghana as a Case Study

The performance of the California Puff (CALPUFF) modelling system has been evaluated using a case study in Ghana. The performance evaluation consisted of a quantitative comparison of dispersion simulation results of SO ₂ and NO ₂ with measurements at the Tema Oil Refinery, and meteorological simulation results with observations from the Tema Meteorological Station, both in the Greater Accra region of Ghana. Four statistical indicators—Index of Agreement (IOA), Fractional Bias (FB), Normalized Mean Square Error (NMSE) and the Pearson correlation coefficient(R) employed in the assessment indicate sufficient reliability of both CALPUFF and its meteorological simulator, CALMET. IOA values of 0.73 and 0.67 and FB values of 1.65 and 1.42 were obtained for SO ₂ and NO ₂ respectively. IOA between measured and modelled emissions were 0.72 and 0.69 for SO ₂ and NO ₂ respectively. The correlations between the simulated and observed emission were 0.66 and 0.08 for SO ₂ and NO ₂ respectively. An IOA value of 0.66 was obtained for both wind speed and wind direction with correlations of 0.29 and 0.58 in comparison with observations from the meteorological station.

• Performance Evaluation
• Emissions
• Dispersion
• Observations

1. Introduction

Dispersion models are useful tools in assessing the impact of emissions to air from a given source. However, in order for model results to be acceptable, the model must be validated to demonstrate that it can produce reliable results for a given modelling scenario.

The kind of data needed for verifying model output depends on the model itself and the users’ needs. For models with meteorological pre-processors or coupled meteorological/chemical models, atmospheric variables observation in some points of the domain are required in order to validate results. Observations can be made at ground level or with a vertical profile, in the case of three dimensional fields. In the case of chemical species concentration, monitoring stations could supply data needed to check model results [1] . [1] - [3] propose some statistical performance measures namely: Correlation Coefficient (R), Mean Bias (MB), Fractional Bias (FB), Normalised Mean Square Error (NMSE), Geometric Mean (GM), Geometric Variance (GV) and Index of Agreement (IOA).

The coefficient of correlation is the measurement of the relationship between observed and predicted values. It indicates the tendency of the predicted values to change with a change in the observed values. A value of R close to unity implies good model performance. The NMSE measures the random spread of the values around the mean and characterizes the amount of deviation between predictions and observations. A good model will have an NMSE value of 0. The IOA reflects the degree to which the observed variable is accurately predicted. The IOA varies from 0 (the theoretical minimum for an inadequate prediction) to 1 (perfect accuracy between the predicted and observed values). The FB is a measure of the systematic bias of the model. It indicates the tendency and the sign of the deviation. A negative FB value indicates model over-prediction and a positive value, an under-prediction.

In most cases, a model is considered acceptable if most of its predictions are within a factor of 2 of the observations [4] [5] . On the other hand, studies by [6] - [8] report that a model can be deemed acceptable if: NMSE ≤ 0.5, −0.5 ≤ FB ≤ +0.5, and IOA > 0.5.

The objective of this study is to assess the performance of the CALPUFF modelling system using measured SO₂ and NO₂ data at the Tema Oil Refinery and meteorological data from the Tema Meteorological Station.

2. Methodology

2.1. Study Area

The study area, capturing Ghana’s only refinery in Tema, the emission source for this study and spanning a domain size of 60 km × 60 km, is located in the Greater Accra Region of Ghana stretching along the Ghanaian Atlantic coast and extending a bit north into Ghana’s interior as shown by Figure 1. The Universal Transverse Mercator (UTM) coordinate of the southwestern corner of the domain in zone 30 are 795 km easting and 600 km northing.

The study area covers the Tema municipal area, parts of the Accra Metropolitan area and extends to parts of the Eastern region. The southern boundary of the metropolis of Accra which coincides with that of the study area is the Gulf of Guinea. Tema is the most industrialised city in Ghana and is locally nicknamed the “Harbour Town” because of its status as Ghana’s largest seaport. It is a major trading center, home to Ghana’s only oil refinery, the Tema Oil Refinery, and other numerous factories, and is linked to Accra by a highway and railway.

2.2. Model Description and Simulation Set-Up

The 3-component CALPUFF modeling system consists of CALMET (a 3-D meteorological model), CALPUFF (an air quality dispersion model) and CALPOST (a post-processing package). CALPUFF is a non-steady-state Lagrangian puff dispersion model, which simulates the effects of time and space-varying meteorological conditions on pollution transport, transformation, and removal. It is capable of modeling instantaneous or continuous releases over distances ranging from tens of meters to hundreds of kilometers. CALMET is a diagnostic meteorological field generator. It combines data from surface stations, upper air stations, over water stations, precipitation stations, with gridded three-dimensional meso-scale data and geophysical data like land use, terrain elevations, to develop fine-scale winds and other meteorological fields consistent with the terrain and land-use resolved on the fine scale grid. The CALMET model can be derived with data either generated by prognostic meteorological models or station observations or combination of both. The three-dimensional meteorological fields generated by CALMET are utilized by CALPUFF to calculate the dispersion of emissions over a specified area. CALPOST is a post-processor designed to average and report concentration results [9] [10] .

Figure 1

Map of study area

In this study, CALMET model was initialised with Weather Research and Forecasting (WRF) data to develop the meteorological field for CALPUFF due to the unavailability of a complete (surface and upper air) meteorological data from local meteorological stations. The WRF model is a fully compressible, non-hydrostatic meso- scale model with a hydrostatic option which uses a terrain-following hybrid sigma-pressure vertical coordinate in its meteorology simulation [11] [12] . Two one-way nested computational domains were set with horizontal grid spacing of 24 km and 12 km representing horizontal grid dimensions of outer and inner domains respectively. The inner domain covered the whole of Ghana within which the study area is located. The model was run with Global Forecast System (GFS) 1 degree reanalysis as initial and boundary conditions with full physics processes included in order to reproduce real meteorological phenomena. Other options include WRF Single Momentum (WSM) 6 Class Graupel for Microphysics Scheme, Rapid Radiative Transfer Model (RRTM) scheme for long wave radiation, Dudhia scheme for short wave radiation, Monin-Obukhov scheme for surface layer, and NOAH model (5 soil layers) for land-surface interactions, while the dynamic options were left as default to produce hourly averages [13] . WRF data for a period of 1 - 31 January, 2008 was generated. CALMET then interpolated and adjusted the coarse resolution of WRF data by accounting for the kinematic effects of higher resolution terrain and land-use. CALMET utilized a divergence-minimization scheme to solve for horizontal components of wind field and similarity theory to extrapolate a vertical profile of wind velocity. The meteorological domain is a 60 km x 60 km with a grid resolution of 1 km.

CALPUFF was then executed with estimated emission rates of SO₂ and NO₂ from the refinery. The CALPUFF model predicts concentrations at specific points or receptors, which are established by the user within the modeling domain. Of the 38 receptor locations specified in the study domain, only one (Tema Oil refinery) also served as a monitoring site [14] . Therefore, evaluation of the dispersion model was performed with measured results from this station.

2.3. Model Evaluation

To validate CALPUFF, measured data of SO₂ and NO₂ emissions made from the 10 - 21 January, 2008 were used. The Differential Optical Absorption System (DOAS) used for the measurements was installed on the premises of the Tema Oil Refinery to monitor and measure the concentration of gases emitted into the atmosphere due to its activities and also due to the operations of other industries within its environs. Details of the set-up and results can be obtained in [15] .

To determine the reliability of the simulation data, verification of simulated values using the WRF and CALMET models was conducted for surface wind speed and direction. Datasets for January 2008 from the Tema Meteorological Station (TMS) were used. Due to the large number of missing data of the other surface wind parameters as well as data format constraints, only wind speed and direction were validated. Figure 2 shows the locations of the TOR and the TMS in the terrain map of the study area.

The statistical verification of model performance in this study was performed using four statistical indicators namely the Index of Agreement (IOA), Fractional Bias (FB), Normalized Mean Square Error (NMSE) and Pearson Correlation Coefficient (R) proposed by [2] [3] which are in agreement with the USEPA guidelines pertaining to model evaluation protocol. The formulae used to derive these indicators are given by:

Figure 2

Terrain Map of the study area showing the refinery and the Tema Meteorological Station

where C_o is the observed quantity, C_p the predicted quantity, C_oi and C_pi are the observed and predicted quantities respectively for N cases.

3. Results and Discussion

3.1. Validation of CALPUFF

Plots of measured and modelled SO₂ and NO₂ are seen in Figure 3 and Figure 4 respectively.

The performance assessment of the model, based on direct quantitative comparisons of observed and predicted mean concentrations as seen in Figure 3 and Figure 4, reveals that CALPUFF predictions for SO₂ were better than NO₂concentrations. From the 10 - 12, 17 - 19 and 21 January, predicted SO₂ concentrations closely approached measurements. Quantitative agreement between predicted and modelled NO₂ concentrations is excellent on 13 January and good for 19 - 21. However, from 13 - 16 January, CALPUFF significantly under-pre- dicted the measured values. Despite these differences between predicted and measured values on some days, the trends in the measurements are accurately predicted especially for SO₂. The simulation results did not take into account SO₂ and NO₂ background concentrations because this data was not available. This may explain the reason for this under-prediction. The measured concentrations are likely to include emissions from other sources which are not considered in the simulations. Another important reason could be the assumption made that emission rates from the refinery are constant. This is unlikely to be representative of actual operating conditions at the refinery.

Figure 3

Plots of measured and modelled SO 2 concentrations

Figure 4

Plots of measured and modelled NO 2 concentrations

It was, however, necessary to make this assumption because information on the hourly, daily and seasonal variations of emission rates was not available. Additionally, possible errors associated with the measurements of SO₂ and NO₂ at the measurement location cannot be overlooked. Results of model performance evaluation after applying the USEPA guidelines pertaining to model evaluation protocol are presented in Table 1.

The index of agreement between predicted concentrations and measurements is better for SO₂ and NO₂. The correlation coefficient of SO₂ with observations is satisfactory but that for NO₂ is negatively weak as shown in Table 1. The NO₂ emissions from automobiles make up a large fraction of ambient NO₂ levels. It is therefore probable that a large fraction of the measurements consists of contributions from the vehicular emission, revealed in the weak correlation and the poor index of agreement. CALPUFF under-predicts NO₂ by a smaller factor than SO₂ as revealed by their FB values. The NMSE values for SO₂ are acceptable whiles that of NO₂ is not. Therefore, based on these indices, the performance of CALPUFF can be described as acceptable.

3.2. Validation of CALMET and WRF Models

Plots of CALMET and WRF outputs with available surface observations (wind speed and direction) for January 2008 from the Tema Meteorological Station (TMS) are shown in Figure 5 and Figure 6. Wind roses generated from the models are also presented in Figure 7 and Figure 8.

It is apparent from Figure 5 that WRF and CALMET models generally under-predicted wind speeds from 9 - 14 January and over-predicted speeds on 16 - 24 of January. However, on the 11, 16, 21 and 23 days, CALMET predictions closely approach observations whiles WRF predicts similar speed on January 15. Figure 6 reveals the over-prediction of wind direction by both models for most part of the period considered. From the other statistical performance measures given in Table 2, it can be observed that both CALMET and WRF share similar indices rating their performances on the same scale with CALMET indicating slightly higher correlation coefficients for both wind direction and speed than WRF. Under-prediction by CALMET is greater than by WRF as seen in the FB values.

The wind rose generated from CALMET as seen in Figure 7, indicated the predominant prevailing wind is south south-westerly (26%) whiles southerly and southwesterly represent about 26%. The remaining winds are spread across a wide spectrum ranging from westerly to north north-easterly. WRF wind rose as presented by Figure 8, predicted an increased percentage of south south-westerly winds (31%) than CALMET. It also indicates 12% of southerly winds and 12% of south-westerly winds which is lower than CALMET. It is clear that the percentages of the other winds are also lower in the WRF wind rose than the CALMET.

The predominant dominant wind speed class predicted by both models is 3.3 - 5.4 m/s with CALMET predicting a higher percentage (75%) than WRF (64%). 30% of the total winds have speeds between 5.4 - 8.5 m/s according to WRF whiles CALMET predicts only 3% of that class. Generally, average wind speeds is higher for

Table 1

Statistical performance indices of the CALPUFF model

Pollutant	R	IOA	NMSE	FB
SO2	0.66	0.73	0.39	0.41
NO2	−0.25	0.36	1.34	0.36

Table 2

Statistical performance indices of CALMET and WRF models

PARAMETER	R	IOA	CALMET NMSE	FB
Speed	0.58	0.66	0.24	0.34
Direction	0.29	0.66	0.02	−0.10
PARAMETER	R	IOA	WRF NMSE	FB
Speed Direction	0.57 0.28	0.65 0.66	0.09 0.02	0.15 −0.10

Figure 5

Plots of observed and modelled wind speeds

Figure 6

Plots of observed and modelled wind direction

Figure 7

Wind rose depicting CALMET surface winds

Figure 8

Wind rose depicting WRF surface winds

WRF than CALMET. Calm winds in both modelled wind (>0.5 m/s) account for only 0.28% of the data. Based on the statistics, it appears that CALMET predictions better approximate observations as it is able to capture small scale effects which cannot be accounted for by WRF. Numerical meteorological models tend to have spatial and temporal phase errors in simulating surface wind. This is because the lowest-level WRF winds must be extrapolated down to ground-level to initialize CALMET. Thus unless the vertical resolution is quite fine, it would be expected that the near-surface CALMET output winds will be biased towards the trends seen in the lowest level of the WRF data, which has a higher elevation in the region compared to the surface stations [16] .

4. Conclusion

The CALPUFF modelling system has been validated using measured concentrations of SO₂ and NO₂ at the Tema Oil Refinery and meteorological data from the Tema Meteorological Station in the Greater Accra Region of Ghana. The statistical measures between observations and simulation results are indicative of the acceptable performance of the dispersion model, CALPUFF, its meteorological component, CALMET and the prognostic WRF model.

Acknowledgements

The authors would like to thank the Abdus Salam International Centre for Theoretical Physics (ICTP), Trieste, Italy for funding the PhD research out of which this article has been extracted, through its Sandwich Training Exchange Programme (STEP). The authors are also grateful to the Tema Oil Refinery for granting permission for the use of emission data for the analysis performed in this paper.

References

1. Martin, R.V. (2008) Satellite Remote Sensing of Surface Air Quality. Atmospheric Environment, 42, 7823-7843. http://dx.doi.org/10.1016/j.atmosenv.2008.07.018

2. Willmott, C.J. (1982) Some Comments on the Evaluation of Model Performance. Bulletin of the American Meteorological Society, 63, 1309-1313. http://dx.doi.org/10.1175/1520-0477(1982)063<1309:SCOTEO>2.0.CO;2

3. Seigneur, C., Pun, B., Pai, P., Louis, J.-F., Solomon, P., Emery, C., Morris, R., Zahniser, M., Worsnop, D., Koutrakis, P., et al. (2000) Guidance for the Performance Evaluation of Three-Dimensional Air Quality Modeling Systems for Particulate Matter and Visibility. Journal of the Air & Waste Management Association, 50, 588-599. http://dx.doi.org/10.1080/10473289.2000.10464036

4. Hanna, S.R., Strimaitis, D.G. and Chang, J.C. (1991) Evaluation of Fourteen Hazardous Gas Models with Ammonia and Hydrogen Fluoride Field Data. Journal of Hazardous Materials, 26, 127-158. http://dx.doi.org/10.1016/0304-3894(91)80002-6

5. Hanna, S., Chang, J. and Strimaitis, D. (1993) Hazardous Gas Model Evaluation with Field Observations. Atmospheric Environment. Part A. General Topics, 27, 2265-2285. http://dx.doi.org/10.1016/0960-1686(93)90397-H

6. Kumar, A., Luo, J. and Bennett, G.F. (1993) Statistical Evaluation of Lower Flammability Distance (LFD) Using Four Hazardous Release Models. Process Safety Progress, 12, 1-11. http://dx.doi.org/10.1002/prs.680120103

7. Zawar-Reza, P., Kingham, S. and Pearce, J. (2005) Evaluation of a Year-Long Dispersion Modelling of Pm10 Using the Mesoscale Model TAPM for Christchurch, New Zealand. Science of the Total Environment, 349, 249-259. http://dx.doi.org/10.1016/j.scitotenv.2005.01.037

8. Ahuja, S. (1996) Evaluation of MESOPUFF-II-Sox Transport and Deposition in the Great Lakes Region. PhD Thesis, University of Toledo.

9. Scire, J.S., Robe, F.R., Fernau, M.E. and Yamartino, R.J. (2000) A User’s Guide for the CALMET Meteorological Model. Vol. 37, Earth Tech, USA, 1-37.

10. Scire, J.S., Strimaitis, D.G. and Yamartino, R.J. (2000b) A User’s Guide for the CALPUFF Dispersion Model. Vol. 521, Earth Tech, Inc., 1-521.

11. Janjic, Z. (2003) A Nonhydrostatic Model Based on a New Approach. Meteorology and Atmospheric Physics, 82, 271-285. http://dx.doi.org/10.1007/s00703-001-0587-6

12. Protonotariou, A., Bossioli, E., Athanasopoulou, E., Dandou, A., Tombrou, M., Assimakopopoulos, V., Flocas, H. and Chelmis, C. (2004) Validation and Inter-Comparison of CALPUFF Regulatory Model to Eulerian Models and Measurements. An Application over the Greater Athens Area, Greece. Proceedings of the 9th International Conference on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes, 131-135.

13. Sackey, S.S. (2012) Development of Differential Optical Absorption System (DOAS) Remote Sensing System for Ground-Based Measurements of Trace Gas Emissions in an Industrial Area. PhD Thesis. University of Cape Coast, Ghana.

14. Affum, H.A., Brunetti, A., Niemela, J.J., Armenio, V., Akaho, E.H.K. and. Danso, K.A. (2015) Preliminary Simulation of Dispersion and Deposition of Refinery Emissions over an Industrial Area in Ghana. The African Review of Physics, 10, 0029.

15. Skamarock, W.C., Klemp, J.B. and Dudhia, J. (2001) Prototypes for the WRF (Weather Research and Forecasting) Model. 9th Conf. Mesoscale Processes, Fort Lauderdale, FL. Amer. Meteorol. Soc., Preprints, J11-J15.

16. Janjic, Z.I. (2002) Nonsingular Implementation of the Mellor-Yamada Level 2.5 Scheme in the NCEP Meso-Scale Model. NCEP Office Note, 437, 61.