I recently had an opportunity to watch a class of 4th graders attempt to take a PARCC test via the TestNav website. I was disappointed by the interface, let alone how the content failed to keep some students on task. Later I googled a sample test to try it out for myself, and was absolutely HORRIFIED at what I was witnessing. Following is a page-by-page report of the TestNav PARCC test, specifically 4th grade ELA (English Language Arts).
tl;dr – TestNav sucks
Starting point: http://parcc.pearson.com/practice-tests/english/
The very fact that Pearson is behind this raises serious red flags for me. My professional experience with Pearson is that they are in it solely to make money. They don’t give a crap about content and/or actual assessments. Again, this is my own take, others may differ.
Page (or screen) 1. The very first sentence does a great job at setting the scene and describing the overall objective of this test. So I read the story and moved on to the question. Or thought I had read the story. Paragraph 18? I only see three paragraphs. Oh, there are 22 “paragraphs” altogether, wow! A bit more reading to do. This is problem #1. How is a child supposed to know there is more to read “below the fold”? There are no instructions to scroll down, and the interface blankly assumes that the user inherently knows how to interpret the scrollbar near the text. Minus one point for Pearson. Problem number two; I spent about 5 minutes or so reading through this story, and this first page is asking two questions about a single word in the entire thing. So now my mind is all caught up about the definition of a single word, and the rest of the story gets flushed out of my short term memory.
Final problem; so I answer two questions. What next. Ahh, yes, the right arrow button, like a web browser. Good thing there are instructions for that (/sarcasm).
Read on to see the other 22 pages and my conclusion…
So the left-hand side of the page didn’t change at all. I state that rather obvious observation because after hitting “next question”, the entire page had to be reloaded. I even had a little “please wait” bar. So I had entire context switch, but half the information has not even changed. The bigger problem, however, is that on the right-hand side, the instructions are totally ambiguous and confusing. “Select three phrases”? How does one even do that? So I “select three phrases” by clicking on them, but what do I do with them? Why are things highlighted? After a little experimentation, I realize that you can click on three “options” for each column and no more (you get a little pop-up toolbox which says you can only select three), which seems to mean that these options are the “answers” I am providing to answer the question.
So now I go back and try to understand what the heck the question is asking of me. I am supposed to find three phrases from the list of five that describe the setting of the story. But this is frustrating because, as everyone knows, the setting is set in the first paragraph. But there are no options available from the first paragraph in the “Phrases that describe the setting”. And even more confusing, the other column called “Evidence” are also phrases that describe the setting. Let’s just choose six (three plus three) random options and move on. This is giving me a headache.
Page 3: We are asked what could possibly be learned from a simple phrase. Given that this is a web front end, they could have linked the phrase to the story, at the very least, given the reader a hint/reminder about where in the story that phrase occurs. But no. What I dislike about this kind of question the most is that it is purely subjective. Sure, we are supposed to respond with the “best” answer, but “best” by whose standard? Did you see a metric posted anywhere? If I were to answer this question, my “performance based assessment” would be based off a total guess. Is this what the test is about? Part B just digs the hole deeper.
Part A asks me for the meaning of two lines from the poem. If they want to know the true meaning, they should call up Maya and ask her. Of the options listed, I could make a reasonable case for two of them. With the question in Part B, only one really matches. I hate these kinds of questions. You have totally destroyed any appreciation I might have had for this poem by forcing me to answer a question the way you want it answered. What about the natural creativity of art, and the ability to see what it means for me? Isn’t that the beauty of poems? Why do I have to abide by what you think the meaning is? What if I disagree with you? Is my “performance based assessment” going to suffer if you are wrong? Additionally, I don’t like it that the lines are not linked. Today’s technology makes it so that if I click on a line in the right-hand side, the left-hand side should automatically scroll to the proper place.
Page 5: Again I am asked for which option I think is best. What if I don’t like any of the options? I have to choose something that someone else put in a list? I could guess what you want and choose randomly. Let’s just go with something that feels right. This is a poem we are talking about afterall. For Part B, I am not finding any of the options that support my answer to Part A. Oh well.
The question for this one is a little more straight forward, and at least there are some instructions about dragging and dropping. But “structural elements” – that sounds a little heavy. Is that the phrase that our 4th grade teachers are using every day to describe various parts of narratives? Hopefully students are familiar with the jargon of both dragging and elements. But the question really has nothing to do with the story nor the poem specifically, but rather prose versus poetry in general. I also found that I could not put all the “structural elements” in the poem box that I wanted to – I was denied dropping in “descriptions”.
Page 7: I am not a big fan of essay questions myself, but the good thing is that now a user can really express themself. I am a little concerned about students that might have a lot of things going on in their head, but have difficulty transposing them via a keyboard. How much time will this consume? Additionally, the little “kitchen sink” of editing tools is very familiar to me, but I question whether many 4th graders will have had a lot of practice using bold, italics and bullet lists that much.
Page 8: Wait a second…. not only has the story changed, but now the objective has changed as well. Since I was keeping a critical eye open for these kinds of things, I noticed it quickly. But there are a number of students who assume the first part has not changed, since for the first 7 pages it was the same. Now we are talking about horses?!?
I do not see much of an issue with Part A, although it would be nice to read the entire context of the phrase. More so for Part B, which cuts and snips individual phrases and leaves the reader with no context at all (still missing the technology where clicking on a phrase highlights the context to the left). Additionally, I was not impressed with the list of choices – none seem to fit my answer that well for Part A. You want me to choose the “best” one? Again, this is going to be a total crap shoot. Especially with the lack of context.
Page 9: Much like the previous page, I found Part A fairly straight forward even though I was not entirely pleased with the list of choices; nothing in Part B backs up my answer from Part A. I also started to question the scientific accuracy of this so-called article. Am I led to believe that normal-sized horses shrunk down the size of ponies in the span of “over 300” short years? It would seem much more reasonable to me that the horses naturally had some predilection for the stark conditions on this island, to which they further adapted in order to survive. But of course, that is not an option I can choose.
Page 10: I personally saw a number of students struggle with this question. The “select three phrases” is similar to page 2 (and just as confusing) but this time the actions when clicking on a phrase are totally different. In fact, I still have no idea what “select three phrases” even means in the context of this question – it seems you can answer this page by completely ignoring those instructions altogether.
The chart we are supposed to complete is not immediately intuitive. I think having gridlines might help, so as to set off what the horse problems are, and how those problems are resolved. The fuzzy science of the article was a bit distracting, but if this is more about reading comprehension, I’ll let it slide for now.
Page 11: Fortunately, not much to say about this one. That’s a good thing, right? 🙂 The story is narrative, has a couple full-color photos and a really grainy, hard-to-understand map of the islands. The Part A question is relatively clear, although I thought the answer for Part B was phrased in such a way as to make it a little counter-intuitive, almost as if they were trying to trick you.
Page 13: I appreciate that Part A is asking about the organization of the narrative, that is a very good question. However Part B is horrible – one sentence to show support for the way an entire piece is organized?
Page 14: Again, we now have two tabs on the left-hand side, although on my screen (which is not small), the two tabs are vertically aligned, so a little less intuitive. On the right-hand side, we have the old “drag and drop” routine, but at least this time the concept is much simpler. I found the instructions still to be confusing – too much verbiage. Also, it is not clear that the first article is derived from another piece of work, unless you read the captions which are annoying aligned vertically so you have to crane your neck to read them.
Page 15: Having a video in the test is a new twist for me. In a classroom full of students, it makes things even more interesting; there is the hassle of getting out earbuds and some children who forget to use earbuds. In the question part, I appreciate that the question specifies a time within the video to note in order to answer the question, but I wonder how many children are familiar with that notation, especially when the question does not mention any sort of units at all (no minutes or seconds)? Do all children know how to jump or skip to a specific section of a video? This is not a test of their web navigation skills.
Page 18: A new objective, a new story, more questions. It is interesting that even though the questions on the right sprawl off the screen, the story on the left has a different containment box. I wonder why that is.
The questions again ask for the “best”. I suppose one might be able to find a “best” for Part A, and possibly for Part B (two choices this time). The good news is that by this time students may be less distracted by the interface and actually able to concentrate on the meat and potatoes.
Page 19: I can appreciate a question about the nature of the shoes in this cute story. However, the instructions say to choose the “best” and the “correct” answers. Wow. If they left off the “correct” part, you never know, students might actually choose incorrect answers. *grin* Which is not hard with multiple-choice selections at all, especially when the mix of answers can be confusing in the first place. Par for the course, Pearson, way to go.
Page 20: Here is my problem with “best” type questions and answers; none of the listed response options are the best. That is my authoritative opinion on the matter. I think the girl is amused and bewildered. But those options are not available for me to choose, so I have to choose something less than best. Ironically, by using the responses in Part B I can figure out which option they want for Part A. But again, this is a test-taking strategy and has absolutely NOTHING AT ALL to do with understanding the story.
Page 22: I see two possible, valid answers for Part A, with a single response from Part B that backs them both up. Additionally, I think this is a really good question to ask and gets folks thinking about how to solve problems and be positive. Hmm….. that might be somewhat lacking in this blog post. 🙂
There are several things which make this just a really bad idea all around:
- Pearson is getting top dollar by inserting themselves as sole providers of state mandated testing technology
- The interface has very little thought given to how a student interacts with the test material itself
- The interface is more distracting than it is helpful; I did not even cover the extra “tools” included in TestNav because, in reality, they are not that helpful for 4th graders – they are just gimmicks
- The test totally sucks at assessment based on performance; it does a much better job at testing how students take tests
- Teachers are spending class time giving “practice” tests so that students can be better prepared to overcome the shortcomings of the testing medium itself (see above points)
- Do not give a PARCC test in our schools (“oh noes, but the state mandates it!” “bah humbug!”)
- Have teachers short-cut the test and assign relevant grades to students based on their immersive and professional experience with each child
- For the love of God, stop contracting with Pearson!!!!
- Petition and campaign against Chris Koch and Susie Morrison (ISBE 866/262-6663 • 217/782-4321), who are at the top level responsible for PARCC being pushed out to our schools