Teardown of PARCC via TestNav

I recently had an opportunity to watch a class of 4th graders attempt to take a PARCC test via the TestNav website. I was disappointed by the interface, let alone how the content failed to keep some students on task. Later I googled a sample test to try it out for myself, and was absolutely HORRIFIED at what I was witnessing. Following is a page-by-page report of the TestNav PARCC test, specifically 4th grade ELA (English Language Arts).

tl;dr – TestNav sucks

Starting point: http://parcc.pearson.com/practice-tests/english/

The very fact that Pearson is behind this raises serious red flags for me. My professional experience with Pearson is that they are in it solely to make money. They don’t give a crap about content and/or actual assessments. Again, this is my own take, others may differ.

pearson_testnav_page1

Page (or screen) 1. The very first sentence does a great job at setting the scene and describing the overall objective of this test. So I read the story and moved on to the question. Or thought I had read the story. Paragraph 18? I only see three paragraphs. Oh, there are 22 “paragraphs” altogether, wow! A bit more reading to do. This is problem #1. How is a child supposed to know there is more to read “below the fold”? There are no instructions to scroll down, and the interface blankly assumes that the user inherently knows how to interpret the scrollbar near the text. Minus one point for Pearson. Problem number two; I spent about 5 minutes or so reading through this story, and this first page is asking two questions about a single word in the entire thing. So now my mind is all caught up about the definition of a single word, and the rest of the story gets flushed out of my short term memory.

Final problem; so I answer two questions. What next. Ahh, yes, the right arrow button, like a web browser. Good thing there are instructions for that (/sarcasm).

 

Read on to see the other 22 pages and my conclusion…

pearson_testnav_page2Page 2: I just realized there are 23 questions. Wow, this is going to take a long time.

So the left-hand side of the page didn’t change at all. I state that rather obvious observation because after hitting “next question”, the entire page had to be reloaded. I even had a little “please wait” bar. So I had entire context switch, but half the information has not even changed. The bigger problem, however, is that on the right-hand side, the instructions are totally ambiguous and confusing. “Select three phrases”? How does one even do that? So I “select three phrases” by clicking on them, but what do I do with them? Why are things highlighted? After a little experimentation, I realize that you can click on three “options” for each column and no more (you get a little pop-up toolbox which says you can only select three), which seems to mean that these options are the “answers” I am providing to answer the question.

So now I go back and try to understand what the heck the question is asking of me. I am supposed to find three phrases from the list of five that describe the setting of the story. But this is frustrating because, as everyone knows, the setting is set in the first paragraph. But there are no options available from the first paragraph in the “Phrases that describe the setting”. And even more confusing, the other column called “Evidence” are also phrases that describe the setting. Let’s just choose six (three plus three) random options and move on. This is giving me a headache.

 

pearson_testnav_page3 Page 3: We are asked what could possibly be learned from a simple phrase. Given that this is a web front end, they could have linked the phrase to the story, at the very least, given the reader a hint/reminder about where in the story that phrase occurs. But no. What I dislike about this kind of question the most is that it is purely subjective. Sure, we are supposed to respond with the “best” answer, but “best” by whose standard? Did you see a metric posted anywhere? If I were to answer this question, my “performance based assessment” would be based off a total guess. Is this what the test is about? Part B just digs the hole deeper.

 

pearson_testnav_page4Page 4: I noticed that the story changed (now a poem), although the overall objective is still the same. Now I have something new to read, and I already know I have to scroll down to read more.

Part A asks me for the meaning of two lines from the poem. If they want to know the true meaning, they should call up Maya and ask her. Of the options listed, I could make a reasonable case for two of them. With the question in Part B, only one really matches. I hate these kinds of questions. You have totally destroyed any appreciation I might have had for this poem by forcing me to answer a question the way you want it answered. What about the natural creativity of art, and the ability to see what it means for me? Isn’t that the beauty of poems? Why do I have to abide by what you think the meaning is? What if I disagree with you? Is my “performance based assessment” going to suffer if you are wrong? Additionally, I don’t like it that the lines are not linked. Today’s technology makes it so that if I click on a line in the right-hand side, the left-hand side should automatically scroll to the proper place.

 

pearson_testnav_page5Page 5: Again I am asked for which option I think is best. What if I don’t like any of the options? I have to choose something that someone else put in a list? I could guess what you want and choose randomly. Let’s just go with something that feels right. This is a poem we are talking about afterall. For Part B, I am not finding any of the options that support my answer to Part A. Oh well.

 

pearson_testnav_page6Page 6: This time, on the left-hand side I see that I now have two “tabs”. Being used to tabs, I know how to use them. But what about the student that does not know how to use them?

The question for this one is a little more straight forward, and at least there are some instructions about dragging and dropping. But “structural elements” – that sounds a little heavy. Is that the phrase that our 4th grade teachers are using every day to describe various parts of narratives? Hopefully students are familiar with the jargon of both dragging and elements. But the question really has nothing to do with the story nor the poem specifically, but rather prose versus poetry in general. I also found that I could not put all the “structural elements” in the poem box that I wanted to – I was denied dropping in “descriptions”.

 

pearson_testnav_page7Page 7: I am not a big fan of essay questions myself, but the good thing is that now a user can really express themself. I am a little concerned about students that might have a lot of things going on in their head, but have difficulty transposing them via a keyboard. How much time will this consume? Additionally, the little “kitchen sink” of editing tools is very familiar to me, but I question whether many 4th graders will have had a lot of practice using bold, italics and bullet lists that much.

 

pearson_testnav_page8Page 8: Wait a second…. not only has the story changed, but now the objective has changed as well. Since I was keeping a critical eye open for these kinds of things, I noticed it quickly. But there are a number of students who assume the first part has not changed, since for the first 7 pages it was the same. Now we are talking about horses?!?

I do not see much of an issue with Part A, although it would be nice to read the entire context of the phrase. More so for Part B, which cuts and snips individual phrases and leaves the reader with no context at all (still missing the technology where clicking on a phrase highlights the context to the left). Additionally, I was not impressed with the list of choices – none seem to fit my answer that well for Part A. You want me to choose the “best” one? Again, this is going to be a total crap shoot. Especially with the lack of context.

 

pearson_testnav_page9Page 9: Much like the previous page, I found Part A fairly straight forward even though I was not entirely pleased with the list of choices; nothing in Part B backs up my answer from Part A. I also started to question the scientific accuracy of this so-called article. Am I led to believe that normal-sized horses shrunk down the size of ponies in the span of “over 300” short years? It would seem much more reasonable to me that the horses naturally had some predilection for the stark conditions on this island, to which they further adapted in order to survive. But of course, that is not an option I can choose.

pearson_testnav_page10Page 10: I personally saw a number of students struggle with this question. The “select three phrases” is similar to page 2 (and just as confusing) but this time the actions when clicking on a phrase are totally different. In fact, I still have no idea what “select three phrases” even means in the context of this question – it seems you can answer this page by completely ignoring those instructions altogether.

The chart we are supposed to complete is not immediately intuitive. I think having gridlines might help, so as to set off what the horse problems are, and how those problems are resolved. The fuzzy science of the article was a bit distracting, but if this is more about reading comprehension, I’ll let it slide for now.

 

pearson_testnav_page11Page 11: Fortunately, not much to say about this one. That’s a good thing, right? 🙂 The story is narrative, has a couple full-color photos and a really grainy, hard-to-understand map of the islands. The Part A question is relatively clear, although I thought the answer for Part B was phrased in such a way as to make it a little counter-intuitive, almost as if they were trying to trick you.

 

pearson_testnav_page12Page 12: In my opinion, this question is the easiest to answer so far. Still would like to have a link that shows me the context of the phrases in the questions.

 

 

 

 

pearson_testnav_page13Page 13: I appreciate that Part A is asking about the organization of the narrative, that is a very good question. However Part B is horrible – one sentence to show support for the way an entire piece is organized?

 

 

 

pearson_testnav_page14Page 14: Again, we now have two tabs on the left-hand side, although on my screen (which is not small), the two tabs are vertically aligned, so a little less intuitive. On the right-hand side, we have the old “drag and drop” routine, but at least this time the concept is much simpler. I found the instructions still to be confusing – too much verbiage. Also, it is not clear that the first article is derived from another piece of work, unless you read the captions which are annoying aligned vertically so you have to crane your neck to read them.

 

pearson_testnav_page15Page 15: Having a video in the test is a new twist for me. In a classroom full of students, it makes things even more interesting; there is the hassle of getting out earbuds and some children who forget to use earbuds. In the question part, I appreciate that the question specifies a time within the video to note in order to answer the question, but I wonder how many children are familiar with that notation, especially when the question does not mention any sort of units at all (no minutes or seconds)? Do all children know how to jump or skip to a specific section of a video? This is not a test of their web navigation skills.

 

pearson_testnav_page16Page 16: As the video is more of an interview, Part A is not too hard to figure out. Part B could be figured out by process of elimination, not necessary from the video at all.

 

 

 

pearson_testnav_page17Page 17: Another essay question. These are probably the best part of the entire test.

 

 

 

 

pearson_testnav_page18Page 18: A new objective, a new story, more questions. It is interesting that even though the questions on the right sprawl off the screen, the story on the left has a different containment box. I wonder why that is.

The questions again ask for the “best”. I suppose one might be able to find a “best” for Part A, and possibly for Part B (two choices this time). The good news is that by this time students may be less distracted by the interface and actually able to concentrate on the meat and potatoes.

 

 

pearson_testnav_page19Page 19: I can appreciate a question about the nature of the shoes in this cute story. However, the instructions say to choose the “best” and the “correct” answers. Wow. If they left off the “correct” part, you never know, students might actually choose incorrect answers. *grin* Which is not hard with multiple-choice selections at all, especially when the mix of answers can be confusing in the first place. Par for the course, Pearson, way to go.

 

pearson_testnav_page20Page 20: Here is my problem with “best” type questions and answers; none of the listed response options are the best. That is my authoritative opinion on the matter. I think the girl is amused and bewildered. But those options are not available for me to choose, so I have to choose something less than best. Ironically, by using the responses in Part B I can figure out which option they want for Part A. But again, this is a test-taking strategy and has absolutely NOTHING AT ALL to do with understanding the story.

 

pearson_testnav_page21Page 21: Same as Page 20; she is neither proud, unsure, worried nor excited. Let’s have a “can you guess what the test writers really want you to say” test?

 

 

 

pearson_testnav_page22Page 22: I see two possible, valid answers for Part A, with a single response from Part B that backs them both up. Additionally, I think this is a really good question to ask and gets folks thinking about how to solve problems and be positive. Hmm….. that might be somewhat lacking in this blog post. 🙂

 

 

pearson_testnav_page23Page 23: And finally the end! Whew! I don’t have any clue how students can be expected to go through this entire test in 60 minutes. Maybe I should write a narrative about it….

 

 

 

 

Conclusion

There are several things which make this just a really bad idea all around:

  • Pearson is getting top dollar by inserting themselves as sole providers of state mandated testing technology
  • The interface has very little thought given to how a student interacts with the test material itself
  • The interface is more distracting than it is helpful; I did not even cover the extra “tools” included in TestNav because, in reality, they are not that helpful for 4th graders – they are just gimmicks
  • The test totally sucks at assessment based on performance; it does a much better job at testing how students take tests
  • Teachers are spending class time giving “practice” tests so that students can be better prepared to overcome the shortcomings of the testing medium itself (see above points)

 

Proposed solutions

  • Do not give a PARCC test in our schools (“oh noes, but the state mandates it!” “bah humbug!”)
  • Have teachers short-cut the test and assign relevant grades to students based on their immersive and professional experience with each child
  • For the love of God, stop contracting with Pearson!!!!
  • Petition and campaign against Chris Koch and Susie Morrison (ISBE 866/262-6663217/782-4321), who are at the top level responsible for PARCC being pushed out to our schools
Advertisements

12 Responses to “Teardown of PARCC via TestNav”

  1. charlesdschultz Says:

    My daughter says the Part B questions were confusing and seemed like a waste of time.

  2. W. Jason Says:

    I set out to take the test, as you did, and report back. I couldn’t get past the first passage, and I really have no idea how to approach, let alone answer, the first set of questions. I’m not going to draw a hard line about how effective it is as an assessment tool, but I will say that I don’t want my kids evaluated this way, because there are so many biases in the method.

    There’s a lot of screen space dedicated to accessibility, screen readers, and so on. This is no surprise, since Illinois is a leader on accessibility measures. In evaluating how well it’s done, there are several experts on the Urbana campus, notably Jon Gunderson and Hadi Esfahani. Hadi hosts monthly exploration discussions, coordinated through Michael McKelvey. They don’t look at content, they look at fairness. Can a student with sight or hearing disabilities get the question, and answer it appropriately? And it goes much deeper beyond that. Regardless of how terrible the actual content of these standardized tests are, these people and their contemporaries deserve a lot of credit for helping differently-abled people get access to better education.

    • charlesdschultz Says:

      W. Jason, you raise a really good point and I thank you for taking the time to provide that perspective here. You asked if TestNav provides a means by which a differently-abled person can get the question and answer it appropriately. Do you happen to know the answer to that question? I confess a near absolutely lack of knowledge about those with different abilities, so your presence here opens a new door in my way of thinking. What environment and what medium typically best serves those with a sight disability? More light? More contrast? I am sure there is a wide range of categories which make up “sight disability”, and thus a spectrum of possibile ways to provide an optimal environment. The hearing disability would certainly be an issue for the lone video in the test, and it seems to me that this could easily be circumvented by having a transcript of the video. I myself have a slight hearing loss, and I am always looking for text versions of audio segments because I find text so much easier to manipulate than audio.

  3. charlesdschultz Says:

    A reader mentioned to me that C-SPAN is carrying a story about the NEA President “taking down” NCLB and talks about standardized tests:
    http://www.c-span.org/video/?323824-4/washington-journal-lily-eskelsen-garcia-education-policy

  4. pattsi Says:

    Folks, I am totally confused–why in the world would any educator/administrator approve of such a negative testing experience for anyone, specifically students. There is plenty of learning negativity that students experience so they do not need more.

    • charlesdschultz Says:

      The State says “jump” and we (the school district administration, in general) have always said “how high?”

      This is exactly why I have contacted the State Superintendent’s office. I have been put in touch with the Director of Assessments, Angela Chamness. I hope to talk to her next week.

      But yes, I am totally confused as well. I have exactly the same exasperation. And I get the feeling that many others share the sentiments.

  5. charlesdschultz Says:

    From Todd Lash’s retweet:
    http://www.washingtonpost.com/blogs/answer-sheet/wp/2015/01/17/test-weary-second-grader-schools-state-school-board-is-that-all-that-matters-to-grown-ups/

    “Hi. My name is Saige Price. I go to Briarwood Elementary School in Florham Park where I attend second grade. Thank you for allowing me to speak today. I would like to talk about play in school and the need for more time for free play. Children should have more recess because it allows us to play with our friends. Instead, we spend most of our time just reading, doing math problems, taking math tests and reading tests.

    Is that all that matters to grown-ups?”

    “I love my teacher but at the beginning of the year I did not want to go to school because I thought school was boring; I still do.”

  6. charlesdschultz Says:

    Apparently, refusing to administer the PARCC test is not an unheard of practice:
    https://teacherbiz.wordpress.com/2015/01/08/newsflash-nj-superintendents-are-allowed-to-accommodate-parcc-refusals/

  7. charlesdschultz Says:

    From another concerned citizen
    Chicago Public Schools decided that a mere 66 of its 600+ school system will get the PARCC test. That certainly says a lot!

    http://www.chicagotribune.com/news/ct-chicago-schools-testing-met-20150116-story.html

  8. charlesdschultz Says:

    I took the Math sample test (4th grade); here are some issues I found:
    – there is no way to make markings on the test, for example, if I wanted to do intermediate work. I would have to use a paper and pencil (or some other medium). Kinda defeats the purpose, doesn’t it?
    – some of the word problems are way too convoluted. I really like how some of the math was very straight forward, but some of the word problems are a pure headache to figure out
    – the included protractor tool is only useful on one question, and even then only slightly
    – there are still some pages where you have to scroll down and there is very little indication that you must do so, other than the scrollbar which is very easy to miss. Very easy. Note that this is not the browser scrollbar, but an imbedded frame.
    – overall, I liked this much more than the ELA test, but still it is obvious that the TestNav website gets in the way of the test itself

  9. Dialog with ISBE on PARCC | Citizen4: A citizen's blog about Champaign Unit 4 Says:

    […] (off-site consultant). It was an amazing exploration of why we are doing PARCC the way we are (see previous post), where we came from and where we are going. But before I go further, both individuals are very […]

  10. Current thoughts on the PARCC | Citizen4: A citizen's blog about Champaign Unit 4 Says:

    […] also took the sample ELA test. You can read that blog post if you like, but in general I was quite agitated – the software itself makes the test more […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: