Pitfalls of Online Testing
There are a number of issues associated with online testing. Below you will find details that we hope will save you time, and provide your students a better learning experience.
1. Technical Problems for Students
Online computer systems should not be used for high stakes unsupervised Tests. There are too many chances of an error on the part of the Internet, the individual's computer, and/or the BbLearn server. It is difficult to separate honest students from dishonest ones who complain. Personal computers lock up and crash at the most inconvenient times, networks get slow and saturated, and server issues occur as well. With so many variables, we suggest that multiple choice Tests be used as just ONE OF MANY grading mechanisms and that students be allowed to drop a low score or even take a Test twice - keeping the higher score. By creating a large question pool, allowing students at least two attempts (asking different questions for each attempt), and recording the highest score you minimize the chances for local issues to cause problems. Combining Tests with papers, discussions, and essays also helps spread out the points over many measurements and diminishes the chance that a student who is honest, but experiences a computer problem, is treated unfairly.
Note: A student may lose Internet access on their computer and not realize it. This is why we discourage the use of the "Force Completion" option which prevents students from resuming an assessment.
2. Timed Tests
Our recommendation is to consider not adding time requirements to tests. Timers are not a valid method of deterring cheating and should not be used for that reason.
3. Enforcing Honesty
Do not expect the BbLearn testing tool to work as an anti-cheating mechanism. It is not designed to foil cheating in an unsupervised environment. Each constraint applied to make cheating more difficult (such as minimum time periods, scripting to prevent copying/printing, etc.) also make submission more difficult for honest students encountering legitimate technical difficulties.
One solution is to move away from multiple choice tests and toward essays, discussions, and papers. Although these alternative assessment methods help, they take time to grade and still do not guarantee honesty. If you do move toward more comprehensive assignments for online courses and stop relying on automated tests, we recommend limiting the number of students per class to 25-30. Be prepared to open additional sections and assign other instructors, if needed.
Another solution is continue to give multiple choice tests, but erect enough barriers (e.g., each student receives randomly selected questions) so that it becomes more cost-effective to devote one's effort to learning rather than cheating. Of course, no system is perfect and the question becomes "how much time and effort do you spend worrying about individuals who choose to spend their time and effort cheating the system rather than learning the material?"
If you choose to deliver unsupervised tests online through BbLearn, please consider the following recommendations:
- Give final exams for a course "in person". BbLearn offers a password option for proctoring all tests taken through it. This is not the least expensive or easiest approach, but is probably the best method for making sure the person who is at the other end of the computer has learned the course material. There is no way in an uncontrolled environment for computer-based testing to prevent cheating. Even with all things working perfectly, a student with two computers can be searching Google for answers or have a past student take the assessment for them. Consider using some of the revenue from fully online classes to offset the costs and coordination of "in person" or proctored final exams.
- Make sure the test questions have been created with Random Order Answering turned on (unless you have an answer like "all of the above"). This displays answers for a question in random order and helps make each Test unique.
- Build Tests using a large set of questions so that no one in the class ends up taking the same test. Large sets of questions, called "Question Sets" in BbLearn, allow you to build tests that are unique for each student. For example, the system randomly will select 10 questions from a set of 50 to deliver to a student. While this doesn't stop someone from taking a test for someone else, it does make it harder for answer keys to be generated and distributed by the student community from semester to semester. If you spend the extra effort to build a very large set of questions and continue to add and subtract questions, your tests will become individualized.
4. Tips for Success
Below are a few tips which, in our experience, help avoid trouble when delivering tests online and are HIGHLY RECOMMENDED:
- Make sure test questions have been created with Random Order Answering turned on.
- Setup your Tests to draw from a large set of questions called a "Question Set", so that every test attempt randomly delivers a different set of questions selected from the available set.
- Allow your students to take the test twice, each time containing a unique set of questions. Then take the highest or average score.
- If the test is timed, allow your students plenty of time to repair or recover from technical problems. Providing double the time you think the test requires allows potential issues to work themselves out, ultimately saving you time. Do not use the "force completion" option as this will not allow students to resume an assessment after a momentary Internet outage.
5. Online Test Guidelines
Below are a few guidelines for assessing your student's progress in an online course: