Using software to fully evaluate case-based open-question exams

20 maart 2012 door ICTO TEAM
Harald Warmelink (TBM)

Implement and test the web application WebLab to implement, administer and grade the case-based open-question exam. This can all lead to professionalization of the examination and save the teachers’ time.

Description of the project

Within the Technology, Policy and Management (TPM) faculty, there is a lot of interest in using software for case-based open-question exams, specifically to implement, administer, grade and evaluate them. This grassroots project initially focused on the last phase (evaluate). However, during the course of the project the focus shifted to the earlier phases (implement, administer and grade), simply because there is limited to no experience with using software for these specific types of exams at all. We used the software WebLab developed by dr. Eelco Visser of the Electrical Engineering, Mathematics and Computer Sciences faculty (see his own grassroots project for more information). This software is a web application. It allows teachers and students to log in using their Net ID and work on an exam digitally. In the first period of the 2012-2013 academic year, WebLab was used to implement, administer and grade the case-based open-question exam of the Master’s course Cross-cultural Management. A total of 129 Master’s students and 3 TPM teachers were involved in the project.

Tips for colleagues

  • Overestimate neither the amount of time digital examination will save, nor the value of saving time on digital examination. WebLab forces you to spend probably more time on designing/implementing an examination. Arguably, that is a good thing. Focus on the time you can save after you have administered the exam (i.e., when you grade and evaluate it).
  • Your students are not all ‘digital natives’. You will get students who want to do the exam on pen and paper instead, for various reasons.
  • Do not drop the formal exam review session/opportunity. WebLab offers students the opportunity to review their answers, grade and the entire answer model at any time. This means that many more students will actually review their exam, leading to a lot more  feedback and complaints. It is best to organize a formal review session to avoid being swamped with e-mails or calls.
  • Make sure your students have ample opportunity to try out the chosen software before they do the exam. In the case of WebLab, this can be done by first using the software for an assignment during the course.
  • Give a lot of thought to the consequences of the chosen software for your exam design. WebLab asks teachers to implement a list of checkable requirements as an answer model. Based on the number of ticks, WebLab automatically calculates the grade for that individual answer. This means that you control how WebLab calculates the grades by determining the amount of requirements an answer must meet and by indicating the weight of each individual question in the entire exam. Make sure you ‘test’ your answer model on paper to determine whether WebLab will calculate the grades befitting the goal of your examination.
  • Make sure the entire exam situation is conform your expectations. TU Delft offers quite some computer halls for digital examination, but they have not been standardized. This means that the computer halls on the TPM faculty are different from the computer halls on the IDE faculty, for example. Make sure you check out the computer hall before you pick it for your digital examination. Support staff can subsequently help you set up a so-called ‘lock-down’ environment to your specifications. You can determine exactly what your students will be able to do on their computers during the examination. You could lock everything down, thus not allowing any internet or network activity. You could also offer access to a single website, e.g. Google Translate or an online dictionary. It is up to you.

Download the poster (pdf) Harald Warmelink made for the Grassroot Market June 2013. 

Pdf containing more indepth information (goals & evaluation)

More information

Docent: Harald Warmelink (TBM); emailadres:
Jaargang: 2012-2013

Terug naar:  Onderwijsthema's | Tools | Docentervaring

© 2019 TU Delft