Assessing Outcomes With Computer-Based vs. Traditional Paper-Based Testing: A Preliminary Study
Abstract
Examining the effectiveness of computer-based testing (CBT) platforms is often limited by software design, user (student) characteristics, unrealistic test development, and/or implementation requirements. The author evaluated CBT vs. traditional paper-based testing (PBT). To optimize the conditions for this comparison, he integrated CBT into a challenging upper-level college course with computer-savvy students using software that puts control over question design and selection in the hands of the instructor. Thirty-three students produced 196 assessments during a semester-long course; statistical analysis revealed no difference between the two assessment modalities. Furthermore, students' preferences were evaluated further by interview and questionnaire; little preference was indicated for either CBT or PBT administration. These results illustrate that student assessments done outside the classroom in a secure computer assisted testing environment can produce results equivalent to more traditional forms of assessment. Although implementing outside-the-classroom computer-based testing (OC-CBT) offers many potential benefits to teachers and students, the human and budgetary resource costs of implementing it can be great. This study offers relevant information to institutions that are considering adopting OC-CBT.
Downloads
Published
Issue
Section
License
Access Agreement Journal on Excellence in College Teaching
Before proceeding you must agree to the terms and conditions of usage as outlined below by clicking on the Accept button and/or by both parties’ signatures below. You will have to do this only once. After agreement, you will be redirected back to the main Journal page. A pdf copy of the terms is available for download.
This Access Agreement (the "Agreement") is effective upon processing of payment ("Effective Date") and is entered into by and between the Journal on Excellence in College Teaching (“JECT”) and the Customer (“Customer").
This Agreement constitutes the entire agreement and supersedes and voids all prior communications, understandings, and agreements relating to the Product(s), including any terms of use displayed to Authorized Users via the online site of the Product(s). Alterations to the Agreement and to any Addendum to the Agreement are only valid and binding if they are recorded in writing and signed by both parties
. I. Definitions
"Authorized Users" shall mean individuals who are authorized by the Customer (which shall include those individuals authorized by the Institutions hereunder) to access the Customer's information services whether on-site or off-site via secure authentication and who are affiliated with the Customer as a student (undergraduates and postgraduates), employee (whether on a permanent or temporary basis), or a contractor of the Customer. Individuals who are not a current student, employee, or a contractor of the Customer, but who are permitted to access the Customer's information services from computer terminals within the physical premises of the Customer ("Walk-In Users"), are also deemed to be Authorized Users, but only for the time they are within the physical premises of the Customer. Walk-In Users may not be given means to access the Product(s) when they are not within the physical premises of the Customer.
"Commercial Use" shall mean use for the purpose of monetary reward (whether by or for the Customer or an Authorized User) by means of the sale, resale, loan, transfer, hire, or other form of exploitation of the Product(s). For the avoidance of doubt, neither recovery of direct cost by the Customer from Authorized Users, nor use by the Customer or Authorized Users of the Product(s) in the course of research funded by a commercial organization, shall be deemed to constitute Commercial Use.
"Educational Purposes" shall mean for the purpose of education, teaching, distance learning, private study and/or research as described in Section II below.
"Institutions" shall mean the Customer's participating institutions, if applicable.
"License" shall mean the non-exclusive, non-transferable right to access and use the Product(s) pursuant to the specific terms and conditions set forth in this Agreement.
"Product(s)" shall mean the products, materials and/or information contained therein that are subject to this Agreement. Product(s) include the Journal on Excellence in College Teaching and the archive of the Learning Communities Journal.
"Reasonable Amount" shall be determined based on guidelines set forth by 17 U.S. Code § 107 (Limitations on exclusive rights, Fair use).
"Secure Authentication" shall mean access to the Product(s) by Internet Protocol ("IP") ranges or by another means of authentication agreed between the Publisher and Customer or Institutions (if applicable) from time to time.
II. Authorized Use of Product(s)
Customer, the Institutions (if applicable), and Authorized Users may use the Product(s) for Educational Purposes as follows:
Analysis. Authorized Users shall be permitted to extract or use information contained in the Product(s) for Educational Purposes, including, but not limited to, text and data mining, extraction and manipulation of information for the purposes of illustration, explanation, example, comment, criticism, teaching, research, or analysis.
Course Packs. Customer, the Institutions, and Authorized Users may use a Reasonable Amount of the Product(s) in the preparation of course packs or other educational materials.
Digital Copy. Customer, the Institutions, and Authorized Users may download and digitally copy a Reasonable Amount of the Product(s).
Display. Customer, the Institutions, and Authorized Users shall have the right to electronically display the Product(s) to the extent necessary to further the intent and purpose of this Agreement.
Electronic Reserve. Customer, the Institutions, and Authorized Users may use a Reasonable Amount of each of the Product(s) in connection with specific courses of instruction offered by Customer.
Interlibrary Loan. The Customer and the Institutions shall be permitted to use Reasonable Amounts of the Content to fulfill occasional requests from other, non-participating institutions, a practice commonly called Interlibrary Loan ("ILL"). Customer and the Institutions shall fulfill such requests in compliance with Section 108 of the United States Copyright Law (17 USC S108, "Limitations on exclusive rights: Reproduction by libraries and archives") and the Guidelines for the Proviso of Subsection 108(2g)(2) prepared by the National Commission on New Technological Uses of Copyrighted Works (CONTU).
The electronic form of the Product(s) may be used as a source for ILL. Customer and the Institutions shall include copyright notices on all ILL transmissions. Notwithstanding anything herein to the contrary, in no event shall any non-secure electronic transmission of files be permitted.
Print Copy. Customer, the Institutions, and Authorized Users may print a Reasonable Amount of the Product(s).
Recover Copying Costs. Customer and the Institutions may charge a reasonable fee to cover costs of copying or printing portions of Product(s) for Authorized Users.
Scholarly Sharing. Authorized Users may transmit to a third party colleague in hard copy or electronically, Reasonable Amounts of the Product(s) for personal use, professional use, or Educational Purposes but in no event for Commercial Use. In addition, Authorized Users have the right to use, with appropriate credit, figures, tables, and brief excerpts from the Product(s) in the Authorized User's own scientific, scholarly, and educational works.
Text Mining. Authorized Users may use the licensed material to perform and engage in text mining/data mining activities for legitimate academic research and other Educational Purposes. Those uses beyond educational use shall require permission from the Publisher.
III. Restrictions
Except as provided herein, the institution shall make reasonable efforts to inform its authorized users not to use, alter, decompile, modify, display, or distribute the Product(s) as follows:
Alter Identification. Remove, obscure, or modify copyright notices, text acknowledging, attributions, or other means of identification or disclaimers as they appear. Alter Product(s).
Alter, decompile, adapt, or modify the Product(s), except to the extent necessary to make it perceptible on a computer screen, or as otherwise permitted in this Agreement. Alteration of words or their order is strictly prohibited.
Commercial Use. No Commercial Use of the Product(s) shall be permitted unless the Customer or an Authorized User has been granted prior written consent by an authorized representative of the Product(s). Use of all or any part of the Product(s) for any Commercial Use or for any purpose other than Educational Purposes.
Distribution. Display or distribute any part of the Product(s) on any electronic network, including without limitation, the Internet, and any other distribution medium now in existence or hereinafter created, other than by a Secure Authentication; print and distribute any portion(s) of the Product(s)s to persons or entities other than the Customer or Authorized Users, except as provided in Section II.
JECT acknowledges that the Customer cannot police or control the actions of its students, faculty, and other Authorized Users with respect to their use of the Product(s). In the event of abuse, the institution shall make prompt and reasonable efforts to heal the breach and notify the publisher.
IV. Term and Termination
This agreement shall commence on the Effective Date and shall remain in effect unless and until terminated as permitted herein (the "Term"). There is no perpetual electronic access to content made available during the term of the agreement.
JECT may terminate this Agreement if Customer violates any of the terms and conditions set forth herein. In the event of any termination of access, JECT will promptly notify the Customer of the basis for termination.
The Customer may terminate this Agreement if sufficient funds are not provided or allotted in future government-approved budgets of the Customer (or reasonably available or expected to become available from other sources at the time the Customer’s payment obligation attaches) to permit the Subscriber, in the exercise of its reasonable administrative discretion, to continue this Agreement.
In the event of any unauthorized use of the Product(s) by an Authorized User, Customer shall cooperate with JECT in the investigation of any unauthorized use of the Product(s) of which it is made aware and shall use reasonable efforts to remedy such unauthorized use and prevent its recurrence. JECT may terminate such Authorized User's access to the Product(s) after first providing reasonable notice to the Customer (in no event less than two (2) weeks) and cooperating with the Customer to avoid recurrence of any unauthorized use. In the event of any termination of access, JECT will promptly notify the Customer
. V. Refunds
In the event that a subscription is canceled by the Customer prior to the subscription end date, the following will be used as guidelines for refunds.
Electronic subscriptions. The Customer shall be entitled to a full refund within 14 days of the start of the most recent subscription term. Refunds requested after 14 days but no later than 60 days from the start of the most recent subscription term will be allowed, minus a 30% processing fee. Refunds will not be granted if requested more than 60 days after the start of the most recent subscription term.
