Talk:Computerized adaptive testing

CAT question
I'm proposing Computer Adaptive Tests for final exams in my University, but I find some coarse reaction. They argument that this kind of tests punishes the most intelligent people and unmotivate them. Has anyone had an experience at implementing CATs for academic courses? -- Mario Montoya (User:200.23.145.23)


 * In order to implement an adpative test, you must first have scaled the pool using item response theory. The sample size typically required for IRT scaling is at least 200 and preferrably several hundreds or thousands of people.  As a consequence, it is uncommon and generally impractical for use in academic settings unless there is a dedicated staff who produce one or a few exams each year and who can manage the CAT.  The way this usually evolves is to first administer the exam on computer for a period of time while the database of questions and answers accumulates.  Then the scaling is performed and an adaptive version of the exam is introduced.


 * I haven't seen any studies that reinforce the notion that CAT's punish the most intelligent students although CAT introduces a new administration method that many students dislike. For example, item review and modification are generally disallowed in CAT because if you get an easier item, then you know that you got the previous item wrong.  If you can backup and correct that answerm then your score will be biased.  In fact, if item review is allowed then here's how you subvert the exam:  Intentionally answer each item wrong until you finish the exam.  The CAT will have served the easiest possible items to you.  Now go back and answer all the easy items correctly.  If you actually get 100% correct, then you should pass but if you don't then you can take them to court and have them explain to a judge why, with 100% correct, you did not pass their exam!  (This is, from memory, Howard Wainer's argument against allowing item review.)  Many students are frustrated that they cannot skip harder questions and return to them later.


 * Why would you want to introduce CAT exams in an academic setting? CAT is really only helpful if you need extremely reliable tests over a broad range of ability (e.g., for categorization or on a broad entrance exam like the GRE).  CAT is too much work and, as Wainer shows, too much of a security risk for general usage. -- Amead

Summary of 1 Dec 2006 edits
Basically, I took all of the references added by Iulus Ascanius and put them in a Wikipedia reference style so readers can link right to the citation. It's a bit messy in code, but it looks nice in the end. I did some minor edits too and added some templates for sections and paragraphs still needing references, but basically that's all I did in the multiple edits I did today. See here for the full edit history. &mdash; Chris53516 (Talk) 21:43, 1 December 2006 (UTC)

Moved page
Do NOT move this page without discussing it first. That is inappropriate. &mdash; Chris53516 (Talk) 02:22, 7 December 2006 (UTC)


 * Sorry about the move - didn't know the rules. I just believe it is more appropriate under the name "Computerized adaptive testing," as this is what is more often called.  A quick scan of the references supports this.  "Computer-adaptive" is not incorrect, just not as common.  Anyone else have thoughts on that (though it doesn't appear that anyone else is contributing material at this point)?


 * I've tried to add more technical information for readers who are interested in the inner workings. Some figures would be helpful, but I'll have to figure out how to do that first.


 * -- Iulus Ascanius 18:11, 13 December 2006 (UTC)


 * I oppose moving this article at all. Computer-adaptive testing is the accepted terminology that appears in research, especially item-response theory research. Any other terminology came from this terminology. &mdash; Chris53516 (Talk) 18:21, 13 December 2006 (UTC)


 * It is hardly "accepted" as the term in research. A complete list of CAT research  has the term "computer-adaptive" used in titles only 43 times - "computerized adaptive" is used approximately 540 times!  And it's impossible to argue with a 'sample' of that size.  Computerized adaptive testing became the accepted terminology when Reckase's use of "tailored testing" died out.
 * Likewise, there is no hyphen in "item-response theory." It is item response theory.
 * --Iulus Ascanius 19:07, 13 December 2006 (UTC)

Moving?
I never came back after the above discussion; I still think that the article should be moved to "Computerized adaptive testing." The website mentioned gives evidence of how much more common that term is. Anyone have an argument to the contrary? Iulus Ascanius (talk) 18:45, 11 February 2009 (UTC)
 * Since no one's objected in almost 2 months, I've gone ahead and moved it.--Aervanath (talk) 16:45, 31 March 2009 (UTC)

External links modified
Hello fellow Wikipedians,

I have just modified 1 one external link on Computerized adaptive testing. Please take a moment to review my edit. If you have any questions, or need the bot to ignore the links, or the page altogether, please visit this simple FaQ for additional information. I made the following changes:
 * Added archive https://web.archive.org/web/20060711174833/http://www.psych.umn.edu:80/psylabs/catcentral/ to http://www.psych.umn.edu/psylabs/catcentral/

When you have finished reviewing my changes, please set the checked parameter below to true or failed to let others know (documentation at ).

Cheers.— InternetArchiveBot  (Report bug) 21:18, 12 November 2016 (UTC)

Computer or computerized?
Should we call it computer adaptive testing or computerized adaptive testing?138.75.40.159 (talk) 18:20, 3 August 2021 (UTC)