Identifying and Explaining Intensification in Prehistoric Fishing Practices IX: Model Evaluation and Parameter Estimates

The previous post in this series introduced the use of mixture models to fit two lognormal distributions to my data on the frequencies of fish caudal vertebra sizes. I have also discussed some technical issues that I encountered while fitting the models. This post presents some results. The following table gives the estimated parameter values for the two lognormal distributions, including the proportion of the assemblage comprised by fish from each distribution, the log means of the two distributions, and the log standard deviations of the two distributions.

Parameter Estimates for Mixture Models

Parameter Estimates for Mixture Models

The log mean and log standard deviation parameters describe the distribution of caudal vertebra height for the fish bone in the modes of smaller (net-caught) and larger (hook- or spear-caught) fish. The estimates of all the parameters also have associated standard errors, but I am still calculating those errors. They will be reported in a later installment in the series.

A future post in the series will also present several theories by which these estimates might be interpreted. For now, I will make a few general observations. Note that the standard deviation of the distribution of smaller fish is consistently greater than the standard deviation of the distribution of larger fish. The standard deviation is scale-dependent, but I can offer an explanation for this observation without standardizing the standard deviations. As explained elsewhere, nets should capture both small and large fish. Small fish were likely more common than large fish. Thus, the distribution of net-caught fish should have a small mean and a relatively large standard deviation. The distribution of fish caught by hook or by spear should have a larger mean and a smaller standard deviation. Hooks and spears would not be likely to catch fish smaller than some threshold size. The estimates support these assumptions. The estimated proportion of net-caught fish is also consistently higher than the proportion of fish caught with other gear. Some variability exists, however, and this variability may be significant.

Having fit the models, another issue must be resolved now. I also need to consider whether these models provide an appropriate fit to the data. In particular, I need to evaluate whether a simpler model might also explain the observed patterns. In this case, an example of a simpler model than the mixture model of two lognormal distributions might be a single lognormal distribution. I fit such single lognormal distributions to the size data from each assemblage using the maximum likelihood method. The negative log likelihoods of the mixture models was consistently lower (showing that it fit the data better) than the negative log likelihoods of the single lognormal distribution models, but this result is not surprising.

Models with many parameters can generally be made to fit data better than models with fewer parameters. Models with fewer parameters, however, should generally be preferred to models with more parameters, following the principle that simpler explanations are better than more complex explanations. Models with many parameters may also be worse at predicting the variability in new data sets. In essence, more complex models may be finely tuned to match the particular, random factors that affected one data set. The next data set will have been affected by those random factors differently. Thus, a simpler model that does not try to “explain” random variation may often do better at predicting additional data. Such models focus on the deterministic factors that pattern variation. These observations

The likelihood ratio test provides a way to compare “nested” models. Models are nested when more complex models can be reduced to simpler models by setting parameters to particular values. In the case of my fish data, I can reduce my mixture model of two lognormal distributions to a single lognormal distribution by setting p=1 or p=0. Recall that p is the proportion of fish in the assemblage that derive from the distribution of mainly smaller (and presumably net-caught) fish.

As the name implies, the likelihood ratio test compares the likelihood values of a complex model and a simpler model. A theorem states that the ratio of these values has a chi-square distribution with degrees of freedom equal to the difference in the number of parameters between the models being compared. Using this theorem, I want to know if the observed ratio attests to a sufficiently significant increase in the fit to the data of the more complex model to justify the added complexity. The following table shows the results of this analysis for my mixture models and the corresponding single lognormal distribution models.

Likelihood Ratio Test Results for Mixture Model and Single Lognormal Distribution

Likelihood Ratio Test Results for Mixture Model and Single Lognormal Distribution

The results provide some support for the use of the mixture models on my data. Many of the p-values for my likelihood ratio tests exceed the arbitrary 0.05 value often employed in studies, although some are lower than this value. Notice that the p-values are generally lower when the sample size is higher. The following scatterplot illustrates this relationship.

Sample size of fish bone and p-value for likelihood ratio tests

Sample size of fish bone and p-value for likelihood ratio tests

P-values often reflect such sample size effects. In addition, no universal threshold exists at which a p-value can be said to truly “significant”. For these reasons, I am comfortable applying the mixture models to all of my assemblages. The mixture models seem sufficiently better at explaining the variability among all of the assemblages to justify the added complexity. I intend to use the mixture models to determine the importance of net-caught fish in each assemblage.

© Scott Pletka and Mathematical Tools, Archaeological Problems, 2009.

Tags: , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: