The Origin of Proficient and Advanced on the WKCE

Here's a link that the Observer has had on his blog for a few years now. He convinced me to self-publish some of my reports to the board and the public in 2007. The topic I wish to discuss right now is the second posting from January 28, 2008 entitled "Student Achievement Update: January 2008."

WKCE (Wisconsin Knowledge and Concepts Exam) is the currently sanctioned testing protocol in Wisconsin to show progress toward the NCLB (No Child Left Behind) goals established in 2002. Each state has their own test and WKCE is ours. A brief review of the report I wrote right after attending the discussion session at the State School Board Convention in January of 2008 will make you conflicted as to whether you should be more astonished that Wisconsin school children can function after high school or that any sane human being with a modicum of statistical skills really expects that 100% of any population can be considered proficient or advanced in all subjects. Proficient has connotations of extreme competency. Which is all the more baffling in this context.

http://dpi.wi.gov/oea/amo.html

This is where I accessed the annual goals for WKCE proficient plus advanced performance.
Annual Measurable Objectives for Reading and Math 2002-03 through 2013-14 are summarized here. One can access the results of any school district by clicking on the Data tab at the top of the page and clicking on WINSS. From there you can view any data for any district you like. (Author's note: If I ever figure out how to save my formatting so I can post tables, really important for my area of expertise, I will be very happy. I will summarize the tables found on the DPI website referenced above because the tables keep losing their formatting. Arrgh.).

PERCENT OF WI STUDENTS WHO NEED TO SCORE AT PROFICIENT/ADVANCED
Annual Measurable Objectives (AMO).

The scores required to be met are listed in order of school year from 2001-2002, the starting point to 2013-2014, each year separated by a comma.

READING: 61%,61%,61%,67.5%,67.5%,67.5%,74%,74%,74%,80.5%,87%,93.5%,100%.
MATH: 37%,37%,37%,47.5%,47.5%,47.5%,58%,58%,58%,68.5%,79%,89.5%,100%.

The proficiency rates are based on the Wisconsin Knowledge and Concepts Examinations (WKCE) and Wisconsin Alternate Assessments (WAA - both for LEP and Students with Disabilities) test scores of students enrolled in the school for a full academic year (FAY). The overall goal is for all Wisconsin students to attain the "Proficient" or "Advanced" levels in Reading and Mathematics by the year 2014.

From these data, one can determine that for the 2009-2010 school year, it is expected that every school in Wisconsin is expected to meet the goal of 74% of their students scoring proficient or advanced in Reading and 58% of the students must score proficient or advanced in Math. Those who do not meet the goals are placed on a list of "failure to meet Adequate Yearly Progress (AYP)." Three years in a row of failing to meet AYP puts your district in jeopardy of losing federal funding. Schools must meet these goals not only in total but by subgroup for which there is a sufficient population in the school, including Disabled Students, Students at Risk and English as a Second Language Students. An editorial comment on these goals is that whoever decided to put these goals in place (Elizabeth Burmaster may have been the Superintendent in charge of the DPI at the time, but I don't know for sure) backloaded these goals in the same way the district backloaded their loan payments in an effort to save pain early on but at the expense of severe pain now. In each case, school officials hedged their bets that the environment would be different and be able to absorb the heavily backloaded goals. In the case of our district loan repayment, it was expected that our enrollment would be so much larger by the time the payments got onerous that the increased state funding would alleviate the tax bills. This has not been the case. (Exhibit A: The mill rate was increased last August by about 10% of its former value. For the record, I didn't vote for it). Now Ms. Burmaster is gone and in the face of imminent statewide failure to meet AYP, Mr. Evers has declared the WKCE an unacceptable measure of student progress in favor of MAP testing. I don't know how fast the new protocols can be implemented, but nothing ever moves quickly in a big bureaucracy, so I don't think change will come soon enough to protect Wisconsin Schools.

Last year, our district had a school (Theodore Robinson Intermediate School or TRIS) placed on the "Failed to meet AYP" list for the first time. As a whole, TRIS has met these goals every year. In the Fall of 2008, the school as a whole scored 80.7% Proficient plus Advanced (P+A) in Reading and 81.5% P+A in Math, well within the goals for that year. Unfortunately, only 29% of the 62 Students with Disabilities scored P+A in Reading and 33.9% in Math that same year, placing us on the failure list because these data were statistically significantly lower than the goal. TRIS put a plan in place, earned a grant to pay for the plan and had stellar success. This year the students with disabilities really boosted their performance on the test for this subgroup to 38.7 % P+A in Reading (nearly 10% improvement!) and 51.6% in Math (nearly 18% improvement). We should remember that even though there were the same number of students taking the test this year as last year, the individual students changed with the advancement of the second graders to third grade and the graduation of the fifth graders out of TRIS. When you’re talking about such a small population, individual differences can be striking. And saddest of all, despite all the good work that has been done at TRIS and great improvement seen, these data still fall statistically short of the NCLB goals for 2009-2010 school year.

Another subgroup that is routinely reviewed for which there is a sufficient population in the Evansville School District is Economically Disadvantaged Students. It is a sad sign of our times that this subgroup has steadily increased every year since 2005. For 2008-2009, TRIS students in this subset technically failed to meet the reading goal of 74% P+A with a score of 71.1%, but applying the standard error to the data kept them in compliance. The math score was higher last year at 73.5% P+A for economically disadvantaged students, well above the goal of 58%. This year the scores did a bit of a flip flop with an increase in Reading to 74.8% P+A in Reading and 68.2% P+A in Math. Both scores meet the goal standards, the Reading being a bit on the shaky side.

I say these data are shaky because next year there are a new, much more rigorous set of standards to meet. The Reading score goal increases to 80.5% of students needing to meet P+A and the math score goal increases to 68.5% P+A. Unless something drastic happens between now and then, both the economically disadvantaged student and the disabled student subgroups will fail to meet AYP at TRIS. It’s possible other schools in our district will also fail, but I have only evaluated the TRIS data at this point. It's also probable that schools all over Wisconsin will begin to fail to meet AYP in the next year or two due to the bar being raised far more than incrementally each year until a miracle occurs and all the children in America will become proficient and advanced in Reading and Math.

As a check point for sanity, let’s evaluate at one of my favorite comparisons with regard to schooling: Girls vs. Boys. In November 2009, the boys at TRIS scored 78.8% P+A in reading and 83.7% P+A in Math. The girls essentially flip these numbers with 84.6% scoring P+A in reading and 79.1% scoring P+A in Math. The old sex stereotypes are still alive and well in our world. And unfortunately, the Reading scores have been showing a steady downward trend for both genders since 2005 and the math scores have been stagnant in the same time frame. The whole school is in danger of failing to meet Adequate Yearly Progress (AYP) next year if the downward trend in reading scores continues and the bar raised to 80.5%. Please note that this is not a failure of the schools, but a failure of NCLB to even pretend to understand statistical data. No population will ever be 100% proficient or advanced unless the definitions of these categories are changed to mediocre and abysmal. They already have been watered down to meaning "passable," according to my report referenced in the link above. This is why the Wisconsin DPI has decided to switch from using WKCE as the measure of AYP to using MAP testing, which stands for Measures of Academic Progress. The students take MAP tests in the fall and in the spring to measure what they have learned in the space of a year of instruction. Our only hope is the implementation BY NEXT YEAR of MAP testing for the measure by which NCLB is judged. Also, we can hope for but not expect some sense to overcome the numbskulls in charge so they can recognize that 100% of a population rarely does anything.

That’s my evaluation of our current and past WKCE data for TRIS. I hope this helps folks understand it a little better.