Non-Scorables
There are 7 reasons an essay may be non-scorable:
1) copied prompt,
2) too short,
3) repetitious,
4) insufficient,
5) off topic, 6) too many unknown words, 7) major syntax errors. Non-scorable status is returned for essays that Intellimetric is just not comfortable returning a score.
An example of an off-topic is an essay writtten by an adult writer or adv proficient writer submitting a response to a middle school prompt. This essay may very well be scorable, however, bares little resemblance to the anchor/training set used to develop the model.
Instead of returning a score, the essay is sent to human readers for evaluation.
Human review is a competitive advantage and a service no other company provides (Pearson, ETS. etc.).
It also supports ongoing review and monitoring of our prompts.
If we see many legitimate off-topics, we can include these in a new anchor/training set.
Typically 2-7% are returned non-scorable.
Teachers/students need to understand that AI engine does not "read" the essay. It is comparing the unknown to the training set.
If atypical it is sent to human readers for review.
In addition, teachers can override or score non-scorables.
Why not use non-scorable as a teachable moment. Have the student get out the rubric and assess their own writing... What would they score? Why--using language of the rubric/my tutor..
Turn off scoring completely and have student work in teams-peer review to assess and provide feedback to the writer (using the rubric)..
Turn on scoring and compare their score to IM.
Do they agree?
Disagree?
This leads to a discussion on scoring accuracy (humans and AI)..
Other ideas: Compare their essays to anchor/training set..
We do display non-scorable reasons.
If a student's essay is non-scorable,
an overview of the reason and instructions are displayed on the report screen and in the portfolio.
We created text for each type of non-scorable.
Off-topic text is displayed below.
Non-Scoreable Reason : This essay was marked as non-scoreable because it MAY be OFF TOPIC. on-Scoreable Reason
An essay that is marked OFF TOPIC MAY not complete the task assigned in the prompt. The details or examples that are used may not clearly relate to the topic and question in the prompt. Even though an essay may be well written, it must answer the question(s) in the prompt.
The scoring center reviews non-scoreable essays and provides feedback. Rather than waiting, however, read over your work and pay attention to the following:
Make sure you have written to the purpose stated in the prompt. Did you write a narrative essay, an informative essay, or a persuasive essay?
Did you do all the parts of the task the prompt asks you to do?
Do you know what audience you are writing to? Are you writing to a friend, someone your age, a teacher, or another adult?
Make sure you have details that support the main idea you are trying to get your reader to understand.
After reading this, you may wish to return to the Assignment Center to revise and resubmit for scoring.