rcrtan

rcrtan provides functions for criterion-referenced test analyses as described in Brown & Hudson (2002). Currently it supports the following item and test analyses:

• Item discrimination

  • Item facility (including IF for passing test takers and failing test takers)
  • B-index
  • Agreement index
  • Item Phi

• Test dependability

  • Subkoviak's (1988) single administration kappa coefficient
  • Subkoviak's (1988) single administration agreement coefficient
  • Brown's (1990) short-cut estimate for phi dependability
  • Brennan's (1984) estimate for phi lambda

Installation & Documentation

Install the most recent version from CRAN

install.packages('rcrtan')

Or, get the development version from github:

install.packages('devtools')
devtools::install_github('gtlaflair/rcrtan')

Item analysis functions

This package contains six unique functions for calculating item statistics and one omnibus function for estimating all of the six unique functions. The formulas have been adapted from Brown and Hudson (2002).

• crt_iteman: omnibus function that estimates item facility, item facility for masters, item facility for non-masters, the B-index, the agreement statistic, and item phi.

• if_total: estimates the item facility values for the entire group of test takers.

• if_pass: estimates the item facility values for the test takers who scored above the cut-score (masters).

• if_fail: estimates the item facility values for the test takers who scored below the cut-score (non-masters).

• b_index: estimates the B-index values for each item. This is the difference in item facility for masters and non-masters.

• agree_stat: estimates the agreement statistic.

• item_phi: estimates item phi.

item_stats <- crt_iteman(data = brown_depend, items = 2:31, raw_cut_score = 21, scale = 'raw')

Test dependability functions

This package contains two functions for estimating test dependability. The formulas have been adapted from Brown and Hudson (2002).

• subkoviak: This function uses Subkoviak's (1988) methods for estimating kappa and rho from a single test administration.

sub_estimate <- subkoviak(data = brown_depend, items = 2:31, raw_cut_score = 21, look_up = FALSE)

• phi_domain: This function uses Brown's (1990) formula for a short-cut estimate of generalizability coefficients for absolute decisions.

phi_estimate <- phi_domain(bh_depend, 2:31)

References

Brennan, R.L. (1984). Estimating the dependability of the scores. In R.A. Berk (Ed.) A guide to criterion-referenced test construction (pp. 292-334). Baltimore: Johns Hopkins University Press.

Brown, J.D. (1990). Short-cut estimates for criterion-referenced test consistency. Language Testing, 7(1), 77-97.

Brown, J.D., & Hudson, T (2002). Criterion-referenced language testing. Cambridge: Cambridge University Press.

Subkoviak, M.J. (1988). A practitioner's guide to computation and interpretation of reliability indices for mastery tests. Journal of Educational Measurement, 25, 47-55.