{smcl}
{* 08jan2008}{...}
{cmd:help mata mm_mgof()}
{hline}

{title:Title}

{p 4 10 2}
{bf:mm_mgof() -- Goodness-of-fit tests for multinomial data}


{title:Syntax}

{p 8 25 2}
{it:real matrix}
{cmd:mm_mgof(}{it:f} [{cmd:,} {it:h}{cmd:,} {it:method}{cmd:,}
{it:stats}{cmd:,} {it:lambda}{cmd:,} {it:nfit}|{it:reps}{cmd:,}
{it:dots}]{cmd:)}


{pstd}
where

{p 12 16 2}
{it:f}:  {it:real colvector} containing observed counts

{p 12 16 2}
{it:h}:  {it:real colvector} containing expected counts (or
probabilities; the scale does not matter)

{p 7 16 2}
{it:method}:  {it:string scalar} containing {cmd:"{ul:a}pprox"}
(large sample chi-squared approximation test; the default),
{cmd:"{ul:m}c"} (Monte Carlo exact test),
or {cmd:"{ul:e}e"} (exhaustive enumeration exact test)

{p 8 16 2} {it:stats}:  {it:string vector} specifying
the test statistics to be used; available statistics are
{cmd:"{ul:x}2"} (Pearson's X2 statistic; the default),
{cmd:"{ul:l}r"} (log likelihood ratio statistic),
{cmd:"{ul:c}r"} (statistic from the Cressie-Read family),
{cmd:"{ul:m}lnp"} (outcome probability statistic),
{cmd:"{ul:k}smirnov"} (two-sided Kolmogorov-Smirnov statistic);
{cmd:"mlnp"} and {cmd:"ksmirnov"} are not allowed with the {cmd:"approx"}
method

{p 7 16 2} {it:lambda}:  {it:real scalar} specifying the lambda parameter for the
Cressie-Read statistic; default is {cmd:2/3}

{p 9 16 2} {it:nfit}:  {it:real scalar} specifying the number of fitted parameters
(imposed restrictions) for the chi-squared approximation test (default is {cmd:0})

{p 9 16 2} {it:reps}:  {it:real scalar} specifying the number of replications
for the {cmd:"mc"} method (default is {cmd:10000})

{p 9 16 2} {it:dots}:  {it:real scalar} causing progress dots to be
displayed with the {cmd:"mc"} or {cmd:"ee"} method


{title:Description}

{pstd}{cmd:mm_mgof()} performs goodness-of-fit tests for discrete
distributions. It returns a matrix containing for each requested statistic
a row with the statistic's value in the first column and the
associated p-value in the second column. {it:f} is a {it:colvector}
containing the observed frequency distribution (i.e. the observed counts
for each category; {cmd:sum(}{it:f}{cmd:)} is the sample size). {it:h} is a
{it:colvector} specifying the null distribution (in counts or proportions)
against which the observed distribution be tested. If {it:h} is omitted or
if {cmd:rows(}{it:h}{cmd:)}==1, the uniform
distribution is used as the null distribution.

{pstd}{it:method} specifies the method used to evaluate the
p-values. Available methods are:

{phang2}{cmd:"approx"} to perform a classic large sample chi-squared
approximation test. {cmd:"approx"} is the default method.

{phang2}{cmd:"mc"} to approximate the exact p-value
by sampling from the null distribution (Monte Carlo simulation). The
proportion of samples in which the test statistic exceeds the
observed statistic gives the p-value. The number of replications (i.e. the
number of drawn samples) is 10000 or as specified by {it:reps}.

{phang2}{cmd:"ee"} to compute the exact p-value by
iterating through all possible data compositions given the
number of observations and the number of categories.
The number of possible compositions grows very fast
with the number of observations and categories (it is equal to
{cmd:comb(}{it:n}+{it:k}-1{cmd:,}{it:k}-1{cmd:)} =
{cmd:comb(}{it:n}+{it:k}-1{cmd:,}{it:n}{cmd:)} =
({it:n}+{it:k}-1)!/(({it:k}-1)!{it:n}!). The {cmd:"ee"} method is
therefore only useful for very small samples with few
categories. An important exception is
when the null distribution is the uniform distribution (and
{it:stats} does not contain {cmd:ksmirnov}). In this case the p-values
are computed based on the partitions of {it:n}. The number of
partitions is typically much smaller than the number of
compositions.

{pstd}{it:stats} specifies the test statistics for which the p-values be
computed. Available statistics are:

{phang2}{cmd:"x2"}: Pearson's X2 statistic

                X2 = sum( ({it:f}-{it:h})^2 / {it:h} )

{pmore2}where {it:f} are the observed counts and {it:h} are the expected counts. X2
is asymptotically chi-square distributed with {it:k}-{it:nfit}-1 degrees
of freedom. {cmd:"x2"} is the default for the {cmd:"approx"} method
({cmd:"approx"} with {cmd:"x2"} is {it:the} classical chi-squared
goodness-of-fit test for multinomial data).

{phang2}{cmd:"lr"}: The log likelihood ratio statistic (or deviance)

                LR = G2 = 2 * sum( {it:f} * ln({it:f}/{it:h}) )

{pmore2}LR is an alternative to Pearson's X2 and is also asymptotically
chi-square distributed with {it:k}-{it:nfit}-1 degrees of freedom.

{phang2}{cmd:"cr"}: The Cressie-Read statistic

                CR = 2/({it:l}*({it:l}+1)) * sum( {it:f} * (({it:f}/{it:h})^{it:l} - 1) )

{pmore2}where {it:l} stands for {it:lambda}, which defaults to 2/3. The Cressie-Read
family includes Pearson's X2 and the LR statistic as special cases
({it:lambda}=1 and {it:lambda}=0, respectively; other special cases are
{it:lambda}=-1/2 for the Freeman-Tukey statistic,
{it:lambda}=-1 for the Kullback-Leibler information, and
{it:lambda}=-2 for Neyman's modified X2 statistic; see Cressie and Read 1984,
Weesie 1997). All members are asymptotically
chi-square distributed with {it:k}-{it:nfit}-1 degrees of freedom.

{phang2}{cmd:"mlnp"}: The statistic -ln({it:p}), where

                {it:p} = {it:n}!/({it:f1}!*...*{it:fk}!) * {it:p1}^{it:f1}*...*{it:pk}^{it:fk}

{pmore2} is the
probability of the observed outcome given the null distribution
({it:n} denotes the sample size, {it:p1},...,{it:pk} are the theoretical
probabilities of the categories,  and {it:f1},...,{it:fk} are the observed
counts). {cmd:"mlnp"}
is not allowed with the {cmd:"approx"} method. {cmd:"mlnp"} corresponds to the
"exact multinomial test", i.e. the computed p-value reflects the exact
probability to observe an outcome that is less probable, given the null
distribution, than the actually observed outcome (see, e.g., Horn 1977,
Cressie and Read 1989).

{phang2}{cmd:"ksmirnov"}: The two-sided Kolmogorov-Smirnov statistic

                D = max( abs({it:H} - {it:F}) )

{pmore2}where {it:H} is the theoretical and {it:F} is the empirical cumulative
distribution function. D is sensitive to the order of the
categories and should therefore only be used with data that has a natural order
(i.e. ordinal or discrete metric data). While the distribution of D
is well known for continuous data, the standard Kolmogorov-Smirnov test
(see help {helpb ksmirnov}) is conservative in the case of discrete
data (see, e.g., Conover 1972). {cmd:mm_mgof()} performs the
Kolmogorov-Smirnov test without making assumptions about
the distribution of D. {cmd:"ksmirnov"} is not available with
the {cmd:"approx"} method.

{pstd}{it:stats} may include several statistics in which case
{cmd:mm_mgof()} returns results for each of the specified statistics. In
the case of the {cmd:"mc"} method, the same set of samples
is used for all specified statistics.

{pstd}{it:dots}!=0 causes progress dots to be displayed with the {cmd:"mc"}
or {cmd:"ee"} method (one dot = 2 percent of computations).

{pstd}See Jann (2008) for a working paper discussing multinomial goodness-of-fit tests
(available from {browse "http://ideas.repec.org/p/ets/wpaper/2.html"}).


{title:Remarks}

{pstd}
Examples:

        {com}: uniformseed(46)
        {res}
        {com}: x = ceil(uniform(7,1)*5) \ 5 \ 5 \ 5 \ 5
        {res}
        {com}: x
        {res}        {txt}1
             {c TLC}{hline 5}{c TRC}
           1 {c |}  {res}5{txt}  {c |}
           2 {c |}  {res}3{txt}  {c |}
           3 {c |}  {res}1{txt}  {c |}
           4 {c |}  {res}4{txt}  {c |}
           5 {c |}  {res}3{txt}  {c |}
           6 {c |}  {res}4{txt}  {c |}
           7 {c |}  {res}5{txt}  {c |}
           8 {c |}  {res}5{txt}  {c |}
           9 {c |}  {res}5{txt}  {c |}
          10 {c |}  {res}5{txt}  {c |}
          11 {c |}  {res}5{txt}  {c |}
             {c BLC}{hline 5}{c BRC}

        {com}: f = mm_freq(x, 1, (1::5))
        {res}
        {com}: (1::5), f
        {res}       {txt}1   2
            {c TLC}{hline 9}{c TRC}
          1 {c |}  {res}1   1{txt}  {c |}
          2 {c |}  {res}2   0{txt}  {c |}
          3 {c |}  {res}3   2{txt}  {c |}
          4 {c |}  {res}4   2{txt}  {c |}
          5 {c |}  {res}5   6{txt}  {c |}
            {c BLC}{hline 9}{c BRC}

        {com}: mm_mgof(f,1,"approx",("x2","lr","cr"))
        {res}       {txt}          1             2
            {c TLC}{hline 29}{c TRC}
          1 {c |}  {res}9.454545455   .0506896631{txt}  {c |}
          2 {c |}  {res}9.700229147   .0457916588{txt}  {c |}
          3 {c |}  {res}9.102749235   .0585819271{txt}  {c |}
            {c BLC}{hline 29}{c BRC}

        {com}: mm_mgof(f,J(5,1,1/5),"approx",("x2","lr","cr"))
        {res}       {txt}          1             2
            {c TLC}{hline 29}{c TRC}
          1 {c |}  {res}9.454545455   .0506896631{txt}  {c |}
          2 {c |}  {res}9.700229147   .0457916588{txt}  {c |}
          3 {c |}  {res}9.102749235   .0585819271{txt}  {c |}
            {c BLC}{hline 29}{c BRC}

        {com}: mm_mgof(f,1,"ee",("x2","lr","cr","mlnp","ksmirnov"))
        {res}       {txt}          1             2
            {c TLC}{hline 29}{c TRC}
          1 {c |}  {res}9.454545455    .057135616{txt}  {c |}
          2 {c |}  {res}9.700229147     .08694016{txt}  {c |}
          3 {c |}  {res}9.102749235    .057135616{txt}  {c |}
          4 {c |}  {res}8.167054764     .08694016{txt}  {c |}
          5 {c |}  {res}.3454545455   .0286034534{txt}  {c |}
            {c BLC}{hline 29}{c BRC}

        {com}: mm_mgof(f,1,"mc",("x2","lr","cr","mlnp","ks"))
        {res}       {txt}          1             2
            {c TLC}{hline 29}{c TRC}
          1 {c |}  {res}9.454545455         .0546{txt}  {c |}
          2 {c |}  {res}9.700229147         .0842{txt}  {c |}
          3 {c |}  {res}9.102749235         .0546{txt}  {c |}
          4 {c |}  {res}8.167054764         .0842{txt}  {c |}
          5 {c |}  {res}.3454545455         .0281{txt}  {c |}
            {c BLC}{hline 29}{c BRC}
        {txt}

{title:Conformability}

    {cmd:mm_mgof(}{it:f}{cmd:,} {it:h}{cmd:,} {it:method}{cmd:,} {it:stats}{cmd:,} {it:lambda}{cmd:,} {it:nfit}|{it:reps}{cmd:)}:
             {it:f}:  {it:k x} 1
             {it:h}:  {it:k x} 1 or 1 {it:x} 1
        {it:method}:  1 {it:x} 1
         {it:stats}:  1 {it:x s} or {it:s x} 1
        {it:lambda}:  1 {it:x} 1
          {it:nfit}:  1 {it:x} 1
          {it:reps}:  1 {it:x} 1
        {it:result}:  max(s,1) {it:x} 2


{title:Diagnostics}

{pstd}{cmd:mm_mgof} aborts with error if {it:h} contains zero or if
{it:f} or {it:h} contain negative
or missing values.

{pstd}{cmd:mm_mgof} aborts with error if {it:f} contains non-integer
values and the method is {cmd:"ee"} or the method is {cmd:"mc"} and the
{cmd:"mlnp"} statistic is requested. The number of observations,
i.e. {cmd:sum(}{it:f}{cmd:)}, is rounded to the nearest integer with the
{cmd:"mc"} method.

{pstd}{cmd:mm_mgof} aborts with error if {it:f} contains zero and the {cmd:"cr"}
statistic is used with {it:lambda}<0. Furthermore, {it:lambda}<0 is only allowed with
the {cmd:"approx"} method.

{pstd}{cmd:mm_mgof}
returns {cmd:(0,1)} for each statistic if {cmd:rows(}{it:f}{cmd:)}<2
or {cmd:sum(}{it:f}{cmd:)}==0.


{title:Source code}

{pstd}
{help moremata_source##mm_mgof:mm_mgof.mata}


{title:References}

{phang} Conover, W. J. (1972). A Kolmogorov Goodness-of-Fit
  Test for Discontinuous Distributions. Journal of the
  American Statistical Association 67: 591-596.

{phang}Cressie, N., T. R. C. Read (1984). Multinomial Goodness-of-Fit Tests. Journal
  of the Royal Statistical Society (Series B) 46: 440-464.

{phang}Cressie, N., T. R. C. Read (1989). Pearson's X^2 and
  the Loglikelihood Ratio Statistic G^2: A Comparative Review. International
  Statistical Review 57: 19-43.

{phang}Horn, S. D. (1977). Goodness-of-Fit Tests for Discrete Data: A
  Review and an Application to a Health Impairment Scale. Biometrics 33: 237-247.

{phang}Jann, B. (2008). Multinomial goodness-of-fit: large sample tests with
survey design correction and exact tests for small samples. ETH Zurich Sociology
Working Paper No. 2. Available from: {browse "http://ideas.repec.org/p/ets/wpaper/2.html"}.

{phang}Weesie, J. (1997). sg68: Goodness-of-fit
statistics for multinomial distributions. Stata Technical
Bulletin Reprints 6: 183-186.


{title:Author}

{pstd} Ben Jann, University of Bern, jann@soz.unibe.ch


{title:Also see}

{psee}
Online:  help for
{helpb ksmirnov},
{helpb mf_mm_freq:mm_freq()},
{helpb mf_mm_subset:mm_subset()},
{bf:{help moremata}}; {helpb mgof} (if installed)