Supporting students’ learning using STACK: Application on a Finance module

Kai-Hong Tee (School of Business and Economics) has been using the STACK question type in Learn to generate mathematical questions. In the post below, Kai explains how it works and why it benefits students. 

Background
I am currently teaching “Financial Management” which is offered as a core module for our second-year students studying for the “Accounting and Financial Management” (AFM) and “Banking and Financial Management” (BFM) degree programs, which are currently one of the best in the UK, with the latest ranking from the Guardian at the second place after University of Bath. The AFM degree is also accredited by the professional accounting body, we follow strict criteria when teaching the students and appropriately assessed them. However, as this module is also opened to other students whose degree programs are less mathematical in nature, such as Marketing and International business, it is important that “Financial Management” must provide sufficient support taking care of a wider range of students of different capabilities. This prompted me to re-think and re-design an existing self-assessment exercise that already has been available to students on LEARN. Figure 1 shows the questions and answers I have been using to help students on Learn. This assessment is based on one topic (equity valuation) in which most students don’t feel very confident, based on my observation and years of experience teaching “Financial Management”.

Figure 1. 

 

As you can see from figure 1, this self-assessment exercise consists of limited questions. The aim, however, is to encourage students to gain a better understanding on the topic through practicing doing the problems. However, for the weaker students, having just 4 or 5 questions may not be enough to develop the skill required for this topic.

Why use the STACK question type?
To efficiently supply larger number of questions so as to allow students to have more chances to practice them to acquire the skills, I adopted the STACK question type in the Learn Quiz activity to enable mathematical questions to be generated automatically. Currently, STACK has been applied in different schools, including the subject areas of engineering and mathematics. Figure 2 shows a snapshot of a question set up using STACK on LEARN. Figure 3 shows that using STACK it is possible to generate a similar question to be done again with different information. This then allows students to have additional practice.

Figure 2.

Figure 3. 

I’ve applied this idea to “financial management”. Figure 4 is an example of a question. Figure 5 shows that students will receive feedback on the answers they provide.

Figure 4.

Figure 5. 

New questions will be generated automatically and can be viewed by clicking on “start a new preview” (see figure 3), the implementation of STACK therefore involves some program coding. Figure 6 gives a snapshot of the “program coding” screen. Here, every question is treated as a “model”, and the “numbers” in the question as “variables”, since they will be different in each attempt of the students to “start a new preview”. Figure 6 shows the range of “variables” (inputs) that you want to set for your questions, representing the “new” question each time a new preview is attempted.

Figure 6.

What’s next?
One issue I face here is that the “new” question won’t have the text changed, just the “numbers”, there may therefore be a familiarity after a few rounds of practicing the questions if there are insufficient questions. Therefore, to increase the effectiveness, more questions can be included to reduce biases arising from “getting right answers because of familiarity”. These questions of different level of difficulties are then re-shuffld for each new round of attempt made by the students.

An area that is worth further developing is to incorporate adaptive learning pathways. Basically, I would only need, for example, 5 questions of different levels of difficulty, and then work on them to create pathways based on the feedback (answers) from the students. From the feedback, this then indicates which level of difficulty is reasonable to further assess the students. It may be possible for STACK to be developed in a way that students are led to attempt questions of reasonable level so that their standards are monitored and matched so that appropriate skills could be developed alongside sufficient practices of more suitable questions. This will be an area for future investigation. If workable, this should support the weaker students better. This implies that students could practice more questions and become more skillful based on their level of understanding, rather than simply practice any available questions without adequate understanding of their standards.

Further information at STACK can be found on the following page:
 http://www.stack.ed.ac.uk 

Tutorials on STACK are available on:  https://stack2.maths.ed.ac.uk/demo/question/type/stack/doc/doc.php/Authoring/Authoring_quick_start.md 

The STACK project team also recently won a HEA CATE award to disseminate STACK across other institutions: https://www.heacademy.ac.uk/person/open-university-stack 

QuickMark(ing)

Focus on feedbackThere is accumulating evidence that on-line marking and feedback can be more effective and efficient than traditional paper-based methods. Thus using technology, to deliver feedback can have several benefits:

  • It allows us to provide more feedback in less time through the use of repeated comments;
  • It helps provide more detailed feedback through the use of in-text annotations and general comments;
  • It can help develop a stronger link between marking criteria, feedback and the grade through the additional use of a rubric.

What is GradeMark?

GradeMark is an on-line marking system which is part of Turnitin (often used to check originality), and can be accessed via Learn. Continue reading

Changes to TurnItIn for next year

The Learn interface to TurnItIn is changing next year.  When the new edition of Learn is released on July 22nd, all TurnItIn assignments will need to be re-created for the new session.  This is no different from previous years, although the software will look slightly different (see: Changes to Turnitin ).

A six-slide PowerPoint presentation is available to show students how to submit coursework using the new TurnItIn assignment.  It may be added to a module as a standalone resource or incorporated into a lecture presentation.

  • If your module uses a TurnItIn assignment activity, a new instance of the activity will need to be created in the 2014-15 module, just as you should have created a new activity last year.
  • The new TurnItIn assignments will be created with a new version of the software, and work through a new TurnItIn account.
  • As usual, students involved in the SAP will use last year’s edition of Learn (which will be called Learn13) and the old 2013 TurnItIn assignments will still work.

TurnItIn for any assignment

TurnItIn have recently announced their Grade Anything initiative, which allows the system to accept files of any type for evaluation.  Obviously the text-matching will only work with files containing text.  However the GradeMark online marking tool is now applicable for a much wider variety of student coursework, as is demonstrated on their video clips .

Don’t get too excited, though – there is no hint that the TurnItIn file size limit of 20 MBytes per file will be increased, so image portfolios or video clips are still likely to be too big to be submitted to TurnItIn.

However one workaround is the ability to use the marking and feedback tools without any submission being present, using a ‘submission template’ (i.e. a blank document).  You might use this to comment on a multimedia submission handed in by some other means, or to assess a live performance e.g. in Drama or Sports Science.

TurnItIn is shifting its emphasis from Originality Checking towards a focus on online marking and feedback provision.  The original functionality is still there and works as it always has done: the ability to handle any file type is an additional feature.

Potential problem using TurnItIn's iPad App

Rob Howe from Northampton University reports a problem with the TurnItIn iPad app which results in loss of data.  The full description is in this blog posting

Essentially, if the iPad user changes their iPad profile during a marking session, they will lose the data already marked, because the iPad thinks it belongs to somebody else.  The data cannot be recovered.

Rob’s advice – to make sure you sync the data often, particularly at the start of a session – seems sound.

 

TurnItIn shifts focus

The September TurnItIn UK User Group meeting was hosted by Leeds Uni this year.
It followed the usual format of a position statement by the host, an overview of the development roadmap and case studies from users.

There are big changes under way at TurnItIn, in response to a massive increase in the demand for their services which have exposed weaknesses in their 10-year old systems. Such is the pace of change that a rolling upgrade programme will see all aspects of the service improved by Spring 2014. Amongst other things, the performance issues that affected some of our users in the summer should be fixed.

The expectation is that by 2016 GradeMark will have taken over from Originality Checking as the main part of the TurnItIn package, so the focus of the software is shifting from Originality Checking towards online marking, without losing any text-matching functionality.

Leeds

It was good to note that Leeds’ policies on Plagiarism are closely aligned with our new Code of Practice (and what happens elsewhere e.g. Bristol and Cranfield).

For example their objectives are
1) To ensure equal vigour in the detection and treatment of plagiarism across all subjects and
2) To provide equal support for students in referencing study skills across all subjects. They have a standard Plagiarism Study Unit which all freshers take in their first semester as part of their tutorial activities.

Development Roadmap

As part of their upgrading effort, TurnItIn are recruiting development programmers in the UK, and we were asked to pass this on to anyone who may be interested in database development work in Newcastle.

  • The TurnItIn iPad app has been well-received and has been updated several times since the initial release so if you are using it, please check for updates.

By Christmas 2013:

  • Colour printing will added to the Document Viewer
  • PowerPoint files will be accepted for Originality Checking
  • Submission to TurnItIn from Goodle Docs or DropBox will be possible

Early 2014:

  • GradeMark gets criterion-based marking without the complexity of a full rubric

Spring 2014:

  • GradeMark will use overlays, which could be used for marking themes e.g. ‘marks for methodology’, ‘marks for analysis’ etc OR for individual tutors.  Visibility of each layer can be controlled, so double-blind marking will be possible for the first time in any online marking tool.

 

The dates above carry the usual health warnings, of course!

The TurnItIn App for iPad

You may have heard that there is a new iPad app for using TurnItIn – including the GradeMark paperless marking tool – available for free from the App Store. This may be of interest to tutors who already have an iPad, and already use GradeMark. It probably isn’t a ‘killer App’ that by itself makes it worth rushing out and buying an iPad.
A key advantage of the app is that you can download the whole class’ assignments to the iPad, and mark them offline, re-syncing when you are back in WiFi range, whereas with the PC version you need to be online all the time.
Almost all of the functionality of the desktop version is available, and some iPad users may find this to be a convenient and quick way of getting marking done in circumstances where it may not otherwise be possible.
The TurnItIn app can be added to all of the Tablets in Teaching iPads, but it will need personal credentials setting up before it can be used.
There are no plans for an Android version.
Setting up such an app with the proper security is always going to be complex, but once set up, the app works very slickly.
Our early experiences indicate:

• The app is set up by default for the US version of TurnItIn, and the iPad setting for the app needs to be changed to TurnItInUK before doing anything else.
• Changes you make can be manually uploaded by re-syncing the iPad, or will automatically be sent if a WiFi connection is available.
• I preferred using the iPad to the iPad Mini because my touches were more accurate and the text was larger and easier to read at the default scaling (you wouldn’t want to have to adjust the display for each assignment you mark, so it’s important that the defaults work well)
• It took 6 ¾ minutes to download 39 essays onto the iPad, so with large cohorts, allow plenty of time for the download.
• If you select the ‘Unlink iPad from TurnItIn’ option, you not only log out of the system, but also delete all of the downloaded assignments. Useful if you are sharing an iPad (does anybody?) but a disaster otherwise.
• Screen rotation (portrait/landscape) works in the normal way. Many screens will re-size using stretch/pinch, but some don’t.

Accessing the submissions
There are two methods of accessing your class’ assignments:

Either

If you are already registered as a TurnItInUK user, you can log in with your email address and TurnItIn password. (Not sure if you’ve been registered already? Use the Retrieve Password link at http://www.submit.ac.uk . If you had a password, you are registered and can use the tool to set a new password. If you are not registered, it will tell you it has failed to find your details.).
Once logged in, you will presented with a list of all your modules, from which you can pick the one with the assignment to be marked.

OR


If you have never been registered as a TurnItInUK user (and most tutors haven’t), you need to:
1. Log into Learn
2. Go to the module
3. Enter the Assignment activity
4. View any one of the submissions by clicking on its Originality Score
5. Once in the Document Viewer, look for a rectangular icon in the bottom left corner. Click the icon to get a 16-character access code for all of the student submissions for this assignment.
6. On the iPad, use this access code to display the submissions for this particular assignment.
7. Because the access code only works once, you’ll need to Sync all the submissions i.e. download them to the iPad, otherwise you’ll need a new access code when you resume marking. If an assignment has multiple markers, each marker will need to get their own access code, and sync the assignments that they have to mark.

Bryan Dawson and Farzana Khandia

Optical Mark Recognition service update

Some colleagues may not be aware that the Optical Mark Recognition (OMR) service continues to be widely used here at Loughborough as a way of delivering exams.

In Semester 1:

• There were 55 OMR exams. 24 were combined question/answer format.
• This represents a 19% increase compared to semester 1 2011-12
• The total numbers of papers printed (including spares) was: 7618
• The total number of OMR exam papers sat was: 6613
• The total number of OMR questions taken and marked was: 332,135

332,135 questions marked automatically… How long would these have taken to mark ‘manually’? If we assume that each question would take, say, 5 seconds, that would equate to 460 hours of marking. So, OMR as a technical approach to assessment still has efficiency savings to offer.

TurnItIn UK User group meeting

This was at the Senate House, London  on February 15th and was attended by 110 delegates representing over 90 different user institutions.

There was a groundswell of disgruntlement over TII’s failures in 2012: at peak times, the service had failed completely on two occasions.

Looking past the spin that was being put on the situation (“There was a failure of communication”:  ‘NO, there was a failure of the service we are paying for.  Better communication wouldn’t have helped…’), the existing TII infrastructure (which was largely unchanged since it was commissioned in 1998) is showing its age, and then unprecedented demand led to the failures.  A major redesign/replacement of the database system is under way.  One benefit of our recession: it’s easier to recruit programmers in the UK than in California, so a lot of the development work is being done over here.

Twitter and RSS feeds are now available giving service status messages, but these had caused problems where there was confusion between TIIUK and TIIworldwide.  Hence although you could have an RSS block on Learn showing the status of the TII service, it might be more nuisance than it is worth.

Jam tomorrow…

Late 2013 – multiple Administrators for an account will be possible.

May 2013 – An  iPad grading app for GradeMark will be available (for free).  This will allow offline marking with automatic syncing when in WiFi range.  There are no plans for an Android equivalent (“Android has too many variations”) or a laptop equivalent.

Q4 2013: Multiple markers will be accommodated by assigning a marking layer to each marker.  Eventually, the layers will be switchable so that marker B can’t see what marker A has already written.  There will only be one mark per submission, with an audit trail of who changed the mark, and when.

Q4 2013: TurnItIn Research – a version geared towards the comparison of research papers rather than student papers.  Will be an extra-cost option and when challenged, Gill Rowell admitted that it didn’t do much that the existing Quick Submit facility did.

Q4 2013: The Document Viewer will be re-written, so expect minor GUI changes.

All TII integrations with VLEs (including the Moodle Direct integration that we use) are being updated to use a new API.  A new plugin, available from Feb 19th will work with Moodle 2.4; work with PeerMark and Rubric Manager; and is supposed to support annual rollover of TII assignments.

In early May, another new Moodle Direct Beta plugin will be available, giving access to all facilities, including a file viewer for the locally-stored submissions.

Other bits…

The ability to draw freehand annotations on a GradeMark submission is not being planned, though it is known that many users would like it.

The iThenticate website (part of the iParadigms suite) lists all the e-Journals which are in the master database.

The e-Rater add-on is an extra-cost option which performs stylistic analysis on the English grammar and syntax of a submission.  It had been rejected by some UK sites because of Data Protection fears (it keeps its own copies of everything submitted to it, and claims to retain the IPR of the submissions).

Electronic Assessment Management group

Many institutions, including Loughborough, are looking at policies around online submission, marking and feedback. Some have already moved to a policy of 100% online submission. Following a recent exchange about this on the national Heads of E-learning Forum (HELF), a new Electronic Assessment Management SIG has been set up, membership of which is open to all in the sector. (I’m on the Steering Group which met for the first time this week.) If you are interested in subscribing to the JISCMAIL list, you can do so at https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=EAM .