Introduction
One of the more popular standards that has emerged in the field of e-assessment is that of Question and Test Interoperability (QTI), developed by the IMS Consortium. The QTI specification describes a data model for representing questions and tests and the reporting of results, thereby allowing the exchange of data (item, test, and results) between tools (such as authoring tools, item banks, test constructional tools, learning environments, and assessment delivery systems). Wide take-up of QTI would facilitate not only the sharing of questions and tests across institutions, but would also enable investment in the development of common tools. QTI is now in its second version (QTIv2), designed for compatibility with other IMS specifications, but despite community enthusiasm there have been only a few real but incomplete examples of QTIv2 being used, and no definitive reference implementation.
At the 2005 JISC/CETIS conference the UK assessment community confirmed that kick-starting the use of the IMS Question and Test Interoperability version 2 specifications was a high priority. However there was little incentive to move towards the new specification as existing public implementations are incomplete and difficult to use. The conference concluded that there needed to be a robust set of tools and services that conformed to the QTIv2 specification to facilitate this migration.
A central function that many systems require is that of rendering questions and responding to users answers. The R2Q2 project produced a core set of web services to provide this functionality.
Question and Test Interoperability
The IMS QTI Specification is a standard for representing questions and tests in XML to allow interchange. A QTI questions is broken down into three parts: ItemBody declares the content of the question itself, ResponseDeclaration declares a variable to store the student’s answer, and OutcomeVariables declares other resulting variables, in this case a score variable to hold the value of the result. In R2Q2 we focus on rendering and responding to the 16 different types of interactions described in version 2 of the QTI specification (QTIv2). These are:
|
1. Choice |
9. Hotspot |
|
2. Order |
10. Select point |
|
3. Associate |
11. Graphic |
|
4. Match |
12. Graphic Order |
|
5. Inline Choice |
13. Graphic Associate |
|
6. Text Entry |
14. Graphic Gap Match |
|
7. Extended Text |
15. Position object |
|
8. Hot Text |
16. Slider |
The list of different question types can be combined with templated questions or adaptive responses, providing an author with numerous alternative methods for writing questions appropriate to the needs of the students. A templated question provides variables in the item body that are instantiated when the question is rendered (for example, inserting different values into the text of maths problems). An adaptive question has a branching structure, and the parts that students see depends on their answers to each part of the branch. Up to sixty-four different combinations are possible.
Service Oriented Architectures
A Service-Oriented Architectures (SOA) allows large complex systems to be modularised in such a way that they are composed of independent software components that offer services to one another through well-defined interfaces. A service approach is ideally suited to loosely coupled systems, where services may be developed by different organisations. This is the philosophy behind JISC’s e-Framework.
Due to the nature of the loose coupling in a SOA, applications can be developed and deployed incrementally. In addition, new features can be easily added after the system is deployed. This modularity and extensibility made an SOA approach especially suitable as a platform for an assessment system with evolving requirements and standards. The R2Q2 architecture implements the SOA of the e-Framework.
R2Q2 Design
The methodology used was an agile approach to design and implementation, which included the FREMA approach to designing Web services. We interviewed experts and stakeholders for a wide picture as to how the R2Q2 service might be used. In addition, we looked at the outputs from previous projects in this area and identified what not to do (lessons learned). An overview of the R2Q2 service requests and responses is shown in Figure 1.
Figure 1 R2Q2 Overview
In the R2Q2 project we provided a definitive render and response processing engine for QTI version 2 question types. This was achieved by taking the requirements and refactoring them such that the main functions were divided between several co-operating Web Services. Figure 2 shows the core functions of the R2Q2 web service.
Figure 2 The R2Q2 Service Architecture
The R2Q2 engine is a loosely coupled architecture comprising of three interoperable services. All the interactions with and within the R2Q2 engine are managed by an internal component called the Router.
The Router is responsible for parsing and passing the various components of the item (QTIv2) to the responsible web services. It also manages the interactions of external software with the system, and it is therefore the only component that handles state.
The Processor service processes the user responses and generates feedback. The Processor compares the user’s answer with a set of rules and generates response variables based on those rules. The Renderer service then renders the item (and any feedback) to the user given these response variables.
We validated the system by using test questions from Graham Smith (these are recognised as being the gold standard for QTIv2). We also used the questions developed as part of the CATS project and questions from E3AN which have be upgraded to version two.
Conclusions
R2Q2 is a definitive response and rendering engine for QTIv2 questions. While this only deals with an Item in QTI terms, it is essential to all processing of QTI questions; that is, it forms the core component of all future systems. Due to the design and use of internal Web services, the system facilitates future enhancement and can be changed to suit any application.
The R2Q2 rendering and response engine of QTIv2 questions is expected to support two main categories of stakeholder:
- Early adopters of QTIv2, who have written questions to this specification and need to validate them. To help these stakeholders we have provided a Web client to which they can submit questions and see the rendered version.
- Other e-Framework Projects . We have provided the core elements of QTIv2 appropriate to a service oriented architecture. Applications in the area of e-assessment, such as question authoring and test delivery, and other aspects of the specification, need to be developed. The R2Q2 project would be an essential element in such future work.
More information, demonstrator and code can be obtained from the website the project Website http://www.r2q2.ecs.soton.ac.uk/.
