I Lied: I Really DO Care

After reading a Wisconsin State Journal article the first week of June regarding which schools failed to achieve Adequate Yearly Progress (AYP), I posted here that I was pretty surprised to see that JC McKenna wasn't on the list. Readers may remember that achieving AYP is required to meet the federal No Child Left Behind law (NCLB). I ranted about liars and statistics and concluded that by 2014 we'll all still be just as stupid as we were in 2001 before the act was implemented. I still believe that but qualify that with a statement that there is plenty of documentation now to tell us the magnitude to which we are ignorant. Unfortunately, that only applies if one believes that the WSAS, an assessment system comprised of the WKCE and the Wisconsin Alternate Assessment for Students with disabilities, accurately measures achievement. I don't have any confidence in the assessment, so my qualification is moot.

If any of you follow my byline in the Review, you will have seen my article regarding Superintendent Carvin's happy announcement on June 13th that all buildings in the district achieved AYP. There was no mention in her announcement that JC McKenna achieved it only through the Confidence Interval qualification. When I wrote my first post on June 8, I was well aware of the statistical qualification of Confidence Intervals (CI). What I wasn't aware of was what level of CI is imposed on the data nor was I privy to the data set on which the CI was calculated. In fact, the DPI seems to be the only ones who have this data set, likely due to concerns about the general population misusing the data.

CI calculations are derived on a basis of the standard deviation (sigma) of a data set, assigning the 75th percent confidence level at +/- one sigma, 95% at +/- two sigma and 99% at +/- three sigma. Use of CI assumes data is normally distributed in a bell curve. Typical experimental evaluations are considered robust at the 95th percent confidence level. The lack of access to the data set used makes it impossible for me to assess the statistical treatment of this data. I am very skeptical that these data are normally distributed, however. Also suspect, in my opinion, is the use of the 99th percent confidence level, which results in a confidence interval through which one could drive a very large truck.

Since I did all of the calculations back in April to determine if any Evansville schools were in danger of failing to meet AYP, I knew JC McKenna missed the 2010-11 reading goal for students with disabilities by more than ten percent. What I didn't expect was Confidence Intervals that exceeded ten percent, but there it was. The online reports issued by the DPI do not have qualifiers on the data. I emailed the DPI to ask what the heck was going on with reports that did not have any qualifiers on them. I was told that the upper limit on the confidence interval around this data statistically predicted that JC McKenna's 70.33% reading proficiency index for students with disabilities has an upper limit of 82.9%, well over the 80.5% required target. Furthermore, the online reports "are space limited" and don't include qualifiers. That is so bogus. The report to school say Yes-CI. Are you telling me that the online reports don't have space for three more characters? They just don't want to field questions from people who have enough brains to figure out what the heck they are doing. Talk about baffling them all with bull%^&.

I am the first person to acknowledge that small populations and wide distributions of ability levels naturally lead to large variability in data. This knowledge is accompanied by the added awareness that these statistical techniques may very well be misused. Lack of access to the raw data from which the calculations are derived makes it impossible to really know. I'm sure that's exactly how the DPI wants it to be.

If the standard 95th percent confidence level had been assigned to the go/no go decision, JC McKenna would have failed to meet AYP. Furthermore, and of greater concern, is that JC McKenna data indicated a significant increase in the number of students in the same subgroup performing at basic and minimum in reading, a step backwards and an indicator of trouble brewing. Combine this with the 10.5% increase in math goal and 6.5% increase in reading goal next year and Evansville will likely see our own school included on the "failed to meet AYP" list for 2011-12. I know you're sick of hearing me predict doom and gloom and then being wrong. If this years trends are repeated next year, and NCLB is not modified, that will be the result.

They will be in a lot of good company to be sure. If next year's increase in goals causes the nearly 100% increase in the number of schools failing AYP that was seen this year, over 400 schools will fail to meet AYP. That rate is nearly one for every district in Wisconsin. I'm not trying to pick on Evansville in general or JC McKenna in particular. School districts are not a fault for the data assessment, but must be wary of accepting the blanket statements from the data manipulators. However, it is not my observation that there is sufficient skepticism on the part of the district administration to aggressively pursue solutions proactively before sanctions are implemented.

The dolts in the federal government need to see that what they have in NCLB is more unfunded mandates that schools simply can't afford to maintain anymore. For the pittance in aid received by our district for this program and others for that matter, it seems time to reevaluate participation in such mandates.