We users do not know the final design of the new test-tool. This makes it difficult to make relevant suggestions. I suggest a community-created whishlist in this post. Hopefully, product-division from ITS can use the whishlist to evaluate the new test-tool. I encourage all users to add whishes as comments. I will then include them in the main list. Also please suggests changes, different ranking, etc. of existing whishes.
Some whishes will naturally be duplicates of excisting ideas. That's ok, this list is meant as a summary.
1. Flexible values for question score and max points
Problem: A question with error often results in an erroneous score. If the question is a multiple choice type, the score cannot presently be changed manually. This might result in a wrong grade on the test, creating false grades in "Status og Oppfølgning". A related problem arises if a question turns out to be irrelevant or illegitimate for a subgroup of students. Presently, questions can only be deactivated for all students or not at all.
Basic wish: Score of any question-type can be set to whatever value within min/max values, or deleted. In addition, max-value can be changed. Any change should be effective for a single student or all students, decided by teacher.
Example: Teacher formulated question badly and in addition marked wrong alternative as correct. Bob misunderstood the question, Lisa anwered correctly. Teacher delete the question for Bob but changes the score from -1 to +2 for Lisa.
2. Easy sharing of tests.
Problem: Today it is difficult to share test outside organizations, for example outside a county in Norway.
Basic whish: Fast and easy sharing of tests nationally. Downloading test as file (as supported presently). Images in question independent of hosting in creators web-files. Score values for questions intact, even if importing to a new test.
Luxury whish: Tests optionally linked to national learning objectives and shared in public database with user ranking system.
3. Print (pdf) support.
Problem: In many situations it is not possible to conduct a test digitally. For example if the school experience network problems, or if PCs are not available for students.
Basic whish: Export digital tests as pdf.
Luxury whish: Tool to automatically process a scanned pdf-version of answered ITS-test and generate results and statistics. Full integration or export-capability of test results to ITS learning, so that student is atomatically assigned his test-scores.
4. Useful statistics for questions and learning objectives.
Problem: Today result-statistics are not useful for questions, and cumbersome to navigate.
Basic whish: See relative scores (opposed to absolute scores available now). Sort questions based on relative score. See mean of all relative scores linked to learning objective. Sort learning objectives based on such value.
Example: Teacher sorts questions from new test based on relative score. He sees that question 15 has 0.05 relative score, and thereby discover unclear framing of the question. He sees that question 42 has relative score of 0.95, and discover this is caused by an intrinsic weakness which makes students able to deduce correct answer logically.
Teacher then sorts learning-objectives based on mean relative scores. He discovers low mean score for learning objective 1. This indicates that the new teaching method used for this objective is unsuitable, or that questions linked to this objective were too difficult.
Luxury whish: Statistics-history for question, learning objective and test.
Example: Teacher has formative test for 5th year in a row, but applied an alterative teaching method for one of the learning objectives. He goes back in the statistics-history of the test and compare previous results with present, and evaluate the new teaching method.
5. Enhanced marking-interface.
Problem: Grading long answers is presently cumbersome. Since they are only partially displayed (even in "Detaljert Vising") teacher must load answer-page, read, and then navigate back to "Detaljert Visning" to submit points.
Basic whish: See all answers for a given question on one page in "array-mode". Give marks with a single click on a "pointruler" next to each answer-cell. Next question in line automatically animate to the middle of the screen. Live statistics-bar shows information about question presently graded.
Example: Teacher click "array"-button to see answers to question 1 for all students. He grades first students answer by clicking once in the middle of a pointruler with values ranging from 1-6 (since max is 6), assigning three points. Immediately, answer from next student (to same question as previously) scrolls to centre automatically. Teacher marks this with 0 points by clicking in the bottom of the pointruler. Next question scrolls into centre, process repeats...
During grading teacher looks at the "live-statiscics-box" in corner and observe the mean score to be 2,2 of 6. He makes a mental note that his grading is perhaps too stringent.
6. Comments on answers
Problem: Today it is not possible to comment student's answer.
Basic whish: Provide comment to answer.
Luxury whish: A message with all comments given in a test is automatically generated and sent to student after grading is finished.
Example: Bob confuses ozon lahyer with greenhouse effect. Teacher clear up misunderstanding in comments field when marking Bob's test. When finishing Bob's test, a message with all comments is automatically sent to Bob.
7. Sound recording as answer
Problem: In many subjects, oral competence is important for final grade. In Norway, for exmaple, oral competence is a "grunnleggende ferdighet". But it can be difficult to find time to hear all students orally in class.
Basic whish: Question type with sound-recorder. Question can also be given as sond-clip
Luxury wish: Sound recorder provide basic editing features.
Example: In question 2 Teacher provide a sound clip with a question. Lisa answer orally using soundrecorder. Teacher grade Lisas answer by assaining points normally.
8. Easy insertion of images independet of web-files.
Problem 1: Presently inserting an image is unusually tedious. a) click image button -> wait 1-2 seconds -> b) click "Last opp og sett inn" button -> wait ~1 second -> c) click "Velg fil" -> d) navigate or select image (>1 click) -> e) click "Last opp og sett inn".
Basic whish: Paste to insert image or "drag image here" -box.
Example: Teacher inserts image from clipboard in qestion by pressing Ctrl-v on keyboard.
Problem 2: Today image is dependent on being hosted in creators web files. This reduce robustness of shared tests.
Basic wish: When an image is used in a test, it is not dependent on user-hosting.
9. No dramatic downsizing of wide images
Problem: Today a wide image will be forcibly downsized to ~480 pixels if it is wider than a certain value (between about 600 or 800). The low resolution makes text and figures difficult to read. Only fix for wide images is to stack text or figures vertically in image editor before use; this is time consuming. Quriously, if adding two images beside each other, pixel-width limitation is not an issue.
Basic wish: Wide images adapts to browser window instead of downsized to low size.
10. Comment page hidden for students
Problem: Pedagogical challenges, information about special sources, information about question difficulty or possible abuse is relevant for teacher using a shared test, but should not be exposed to students.
Basic wish: Comment field in tests hidden for students.
(Edit 4.5 Clarified language and added examples. Renamed from chekc-list to whishlist)
(Edit 5.5 added wish 6 and 7)
(Edit 6.5 added wish 8 and 9, made minor clarifications elsewhere)
(Edit 7.5 added wish 10)
|Category||Grading and assessment|
Hi Frank and everyone else interested in the new test tool
The test tool is one of the most complex parts of itslearning, and rewriting the code has turned out to be a nice “little” challenge for us. We’re super keen on getting this right, and working hard to finalizing it. The new main focus for the new test tool is to make it mobile friendly and shareable. I don’t dear to get into details because the design still might change.
We always tweak existing and new features based on user feedback and research. We’ll do the same with the new test tool. In other words: we hope that you all will help us with a lot of feedback (good and bad) after it’s been released.
Most of the ideas in this “wish-list” exist as separate ideas, and I kindly ask you all to please post one idea at a time (I understand this wish list is meant as a collection of already posted ideas). Please see my replies to previously posted ideas. The “wish-list” is of cause presented to the team working with the new test tool.