"High Achievement always takes place in the framework of high expectation." - Charles Kettering

Thursday, May 13, 2010

The Origin of Proficient and Advanced on the WKCE

Here's a link that the Observer has had on his blog for a few years now. He convinced me to self-publish some of my reports to the board and the public in 2007. The topic I wish to discuss right now is the second posting from January 28, 2008 entitled "Student Achievement Update: January 2008."

WKCE (Wisconsin Knowledge and Concepts Exam) is the currently sanctioned testing protocol in Wisconsin to show progress toward the NCLB (No Child Left Behind) goals established in 2002. Each state has their own test and WKCE is ours. A brief review of the report I wrote right after attending the discussion session at the State School Board Convention in January of 2008 will make you conflicted as to whether you should be more astonished that Wisconsin school children can function after high school or that any sane human being with a modicum of statistical skills really expects that 100% of any population can be considered proficient or advanced in all subjects. Proficient has connotations of extreme competency. Which is all the more baffling in this context.


This is where I accessed the annual goals for WKCE proficient plus advanced performance.
Annual Measurable Objectives for Reading and Math 2002-03 through 2013-14 are summarized here. One can access the results of any school district by clicking on the Data tab at the top of the page and clicking on WINSS. From there you can view any data for any district you like. (Author's note: If I ever figure out how to save my formatting so I can post tables, really important for my area of expertise, I will be very happy. I will summarize the tables found on the DPI website referenced above because the tables keep losing their formatting. Arrgh.).

Annual Measurable Objectives (AMO).

The scores required to be met are listed in order of school year from 2001-2002, the starting point to 2013-2014, each year separated by a comma.

READING: 61%,61%,61%,67.5%,67.5%,67.5%,74%,74%,74%,80.5%,87%,93.5%,100%.
MATH: 37%,37%,37%,47.5%,47.5%,47.5%,58%,58%,58%,68.5%,79%,89.5%,100%.

The proficiency rates are based on the Wisconsin Knowledge and Concepts Examinations (WKCE) and Wisconsin Alternate Assessments (WAA - both for LEP and Students with Disabilities) test scores of students enrolled in the school for a full academic year (FAY). The overall goal is for all Wisconsin students to attain the "Proficient" or "Advanced" levels in Reading and Mathematics by the year 2014.

From these data, one can determine that for the 2009-2010 school year, it is expected that every school in Wisconsin is expected to meet the goal of 74% of their students scoring proficient or advanced in Reading and 58% of the students must score proficient or advanced in Math. Those who do not meet the goals are placed on a list of "failure to meet Adequate Yearly Progress (AYP)." Three years in a row of failing to meet AYP puts your district in jeopardy of losing federal funding. Schools must meet these goals not only in total but by subgroup for which there is a sufficient population in the school, including Disabled Students, Students at Risk and English as a Second Language Students. An editorial comment on these goals is that whoever decided to put these goals in place (Elizabeth Burmaster may have been the Superintendent in charge of the DPI at the time, but I don't know for sure) backloaded these goals in the same way the district backloaded their loan payments in an effort to save pain early on but at the expense of severe pain now. In each case, school officials hedged their bets that the environment would be different and be able to absorb the heavily backloaded goals. In the case of our district loan repayment, it was expected that our enrollment would be so much larger by the time the payments got onerous that the increased state funding would alleviate the tax bills. This has not been the case. (Exhibit A: The mill rate was increased last August by about 10% of its former value. For the record, I didn't vote for it). Now Ms. Burmaster is gone and in the face of imminent statewide failure to meet AYP, Mr. Evers has declared the WKCE an unacceptable measure of student progress in favor of MAP testing. I don't know how fast the new protocols can be implemented, but nothing ever moves quickly in a big bureaucracy, so I don't think change will come soon enough to protect Wisconsin Schools.

Last year, our district had a school (Theodore Robinson Intermediate School or TRIS) placed on the "Failed to meet AYP" list for the first time. As a whole, TRIS has met these goals every year. In the Fall of 2008, the school as a whole scored 80.7% Proficient plus Advanced (P+A) in Reading and 81.5% P+A in Math, well within the goals for that year. Unfortunately, only 29% of the 62 Students with Disabilities scored P+A in Reading and 33.9% in Math that same year, placing us on the failure list because these data were statistically significantly lower than the goal. TRIS put a plan in place, earned a grant to pay for the plan and had stellar success. This year the students with disabilities really boosted their performance on the test for this subgroup to 38.7 % P+A in Reading (nearly 10% improvement!) and 51.6% in Math (nearly 18% improvement). We should remember that even though there were the same number of students taking the test this year as last year, the individual students changed with the advancement of the second graders to third grade and the graduation of the fifth graders out of TRIS. When you’re talking about such a small population, individual differences can be striking. And saddest of all, despite all the good work that has been done at TRIS and great improvement seen, these data still fall statistically short of the NCLB goals for 2009-2010 school year.

Another subgroup that is routinely reviewed for which there is a sufficient population in the Evansville School District is Economically Disadvantaged Students. It is a sad sign of our times that this subgroup has steadily increased every year since 2005. For 2008-2009, TRIS students in this subset technically failed to meet the reading goal of 74% P+A with a score of 71.1%, but applying the standard error to the data kept them in compliance. The math score was higher last year at 73.5% P+A for economically disadvantaged students, well above the goal of 58%. This year the scores did a bit of a flip flop with an increase in Reading to 74.8% P+A in Reading and 68.2% P+A in Math. Both scores meet the goal standards, the Reading being a bit on the shaky side.

I say these data are shaky because next year there are a new, much more rigorous set of standards to meet. The Reading score goal increases to 80.5% of students needing to meet P+A and the math score goal increases to 68.5% P+A. Unless something drastic happens between now and then, both the economically disadvantaged student and the disabled student subgroups will fail to meet AYP at TRIS. It’s possible other schools in our district will also fail, but I have only evaluated the TRIS data at this point. It's also probable that schools all over Wisconsin will begin to fail to meet AYP in the next year or two due to the bar being raised far more than incrementally each year until a miracle occurs and all the children in America will become proficient and advanced in Reading and Math.

As a check point for sanity, let’s evaluate at one of my favorite comparisons with regard to schooling: Girls vs. Boys. In November 2009, the boys at TRIS scored 78.8% P+A in reading and 83.7% P+A in Math. The girls essentially flip these numbers with 84.6% scoring P+A in reading and 79.1% scoring P+A in Math. The old sex stereotypes are still alive and well in our world. And unfortunately, the Reading scores have been showing a steady downward trend for both genders since 2005 and the math scores have been stagnant in the same time frame. The whole school is in danger of failing to meet Adequate Yearly Progress (AYP) next year if the downward trend in reading scores continues and the bar raised to 80.5%. Please note that this is not a failure of the schools, but a failure of NCLB to even pretend to understand statistical data. No population will ever be 100% proficient or advanced unless the definitions of these categories are changed to mediocre and abysmal. They already have been watered down to meaning "passable," according to my report referenced in the link above. This is why the Wisconsin DPI has decided to switch from using WKCE as the measure of AYP to using MAP testing, which stands for Measures of Academic Progress. The students take MAP tests in the fall and in the spring to measure what they have learned in the space of a year of instruction. Our only hope is the implementation BY NEXT YEAR of MAP testing for the measure by which NCLB is judged. Also, we can hope for but not expect some sense to overcome the numbskulls in charge so they can recognize that 100% of a population rarely does anything.

That’s my evaluation of our current and past WKCE data for TRIS. I hope this helps folks understand it a little better.


Fred said...

Hello Melissa,

It's clear you have VERY high expectations for your readers. This post so long & convoluted that it's very hard to read and understand. Parse this 73-word sentence:

"A brief review of the report I wrote right after attending the discussion session at the State School Board Convention in January of 2008 will make you conflicted as to whether you should be more astonished that Wisconsin school children can function after high school or that any sane human being with a modicum of statistical skills really expects that 100% of any population can be considered proficient or advanced in all subjects. "

I actually found the linked report, (hard enough) and my eyes sort of glazed over about two paragraphs in. But I persisted, and I
I think I finally got to the point of the report: the tests in question serve little purpose save getting Federal aid.

Back to the recent post: It seems your real point here is that NCLB is royally screwed up, and that achieving their standards is gonna be impossible. Ergo, bye-bye federal dollars for TRIS.

What reasonably sane human being is going to read all this to try to ease out your main message? It may be that you've got some great ideas, but I for one am really reluctant to try to find the needle in any of the haystacks your writing has created.

As a friendly suggestion, consider boning up on your writing skills -- perhaps starting with Strunk & White. (look it up.)

By the way, I think that E'ville schools' grading standards give our students an inflated view of their academic abilities -- regardless of the standardized tests.

Katy said...

Hi Fred:
Thanks for the feedback. If you read the previous post regarding co-curricular participation and standards, I addressed the grade inflation issue a bit as well. The new principal is trying to get a handle on this, but it won't be easy or popular.

chasinthenews said...

I would like to see Fred come to a school board meeting. I have never seen him at one. Melissa explains things very well. If you can not understand her explanation, I don't know whose explanation you would understand.

Anonymous said...

I too get a bit lost in your analysis. While I think there is tremendous value in dissecting the data, it's also worth taking an overview, and comparing how we handle the task of educating our children to other districts within our state. Schooldiggerdotcom ranks Evansville Community School District at number 239 out of 419. That's the bottom 43%. Only 180 school districts perform below us in educating their students. TRIS ranked 590 out of 1054. Which falls in line at the bottom 44%.

This is appalling, and the School board should be outlining to the community their plan to improve their performance. Nearly 60% of the State is doing a better job.

Katy said...

Thanks for the comment, anonymous! The link you suggested is useful in other ways as well. For instance, it indicates that TRIS and EHS both moved up in their rankings from the previous year. TRIS moved up 48 spots from 638/1048 to 590/1048 while the high school moved from 262/464 to 191/474. JC McKenna Middle School moved down from 202/571 to 244/571 in the same time frame.

One thing the school board did two years ago was to bring back the curriculum coordinator position it got rid of in 2002 or 2003 to save money. A brief view of historic WKCE performance suggests that eliminating the curriculum coordinator has had a deleterious effect on our student's WKCE performance. While the state performance is inching up year by year, our district has shown consistent reduction in performance. Beginning two years ago, Evansville schools scored below the average state performance on several measurement standards. This was the first time in a while that this has occurred. This was probably the tipping point at which students who learned with the benefit of a curriculum coordinator entered the higher grades or graduated and the younger students were floundering. Ms. Landers is an excellent curriculum coordinator and has already taken steps to correct this decline. I expect it will take several time to see significant results.

Anonymous said...

I'm not following the logic you're using here. You post that reading scores have been showing a steady downward trend for both genders since 2005. You state that beginning two years ago, Evansville schools scored below the average state performance on several measurement standards. Yet this is the time the board brought back the CC position. It doesn't seem to follow that the added expense to the district is as clear a benefit to the students as you imply.

Elsewhere in your blog, you post of the downward impact to the averages the special ed kids have on the test scores. So there are a number of issues at work here that contribute to our poor showing.

The Board should ask the district to perform an honest assessment of why it is we are in the lower 43 percentile of the State. Until we can identify the problems, we can't expect to see them fixed.

Katy said...

The WKCE data decline over the last two years comes as a result of neglecting curriculum direction for the previous six years. The curriculum changes that have been implemented over the last two years will take time to show impact in WKCE improvement. There can't be an expectation of immediate improvement from day one of the new curriculum director when 6 years of neglect have to be overcome. The lack of logic here is from a person expecting instant improvement in test scores when curriculum alignment has been neglected for so long.