JATT Launches Volume 21 with Three New Articles and New Call for Papers

• catIRT tools: A "Shiny" Application for Item Response Theory Calibration and Computerized Adaptive Testing Simulation
• Developing Authentic Digital Math Assessments
• The Impact of Technology-Enhanced Items on Test-Taker Disengagement

Co-editors Dianne Henderson of ACT and Holly Gardiner of Kaplan kicked off The Journal of Applied Testing Technology's 2021 Volume with three new articles focusing on computer adaptive testing, digital assessment and technology enhanced items.  

catIRT tools: A "Shiny" Application for Item Response Theory Calibration and Computerized Adaptive Testing Simulation aims to introduce catIRT tools which facilitates researchers’ Item Response Theory (IRT) and Computerized Adaptive Testing (CAT) simulations. catIRT tools provides an interface for mirt and catR packages through the shiny package in R. Through this interface, researchers can apply IRT calibration and CAT simulations although they do not have any coding skills. Dichotomous and polytomous IRT models are supported in IRT calibration and Yen’s Q3 statistics is calculated for the estimation of local independence. In CAT simulation, researchers can use their own parameters and responses, and can produce items or responses. In addition to several item selection and ability estimation methods, researchers can also decide on the specific stopping rule to be used.

Developing Authentic Digital Math Assessments focuses on the shift to next generation digital assessments in which increased attention has focused on Technology-Enhanced Assessments and Items (TEIs). This study evaluated the feasibility of a high-fidelity digital assessment item response format, which allows students to solve mathematics questions on a tablet using a digital pen. This digital ink approach allows students to hand write their responses while showing their work in a digital format. Responses obtained using the digital pen were compared to both paper- and type-written response formats in a repeated measures design. Results showed that students liked the digital pen format and preferred it to the type-written format, but continued to rank the paper format as their first preference. Statistical comparisons showed no significant differences in student performance across response modalities, but means fell in the predicted order favoring digital pen over keyboard for the digital response formats. Results are discussed in light of software usability, technical issues during data collection, and statistical power for detecting effects in the study.The

Impact of Technology-Enhanced Items on Test-Taker Disengagement  examines how Technology-Enhanced Items (TEIs) have been purported to be more motivating and engaging to test takers than traditional multiple-choice items. The claim of enhanced engagement, however, has thus far received limited research attention. This study examined the rates of rapid-guessing behavior received by three types of items (multiple-choice, multiple select, and TEIs) on a commonly-used K-12 adaptive achievement test. Across three subject areas, the TEIs consistently showed the lowest rapid guessing rate, suggesting that their use may help mitigate the problem of disengaged test taking.

To read these articles in their entirety visit the JATT website.

New Call for Papers Opens March 30, 2021