(Part IV) Kasich's school rankings exeplify ignorance about K-12 education....

From John Curry, December 4, 2011

http://www.plunderbund.com/

Kasich's school rankings exemplify ignorance about K-12 education

By Greg On November 26, 2011

Back in April, when Kasich’s budget was still being analyzed, we posted about the Governor’s proposed criteria for ranking Ohio’s public school districts and how that ranking connected to his proposed school funding numbers. Our analysis was based on the budget bill (HB153) as it existed at that current time, a listing of five criteria applied to public school districts. The budget was modified between that time and its final passage, adding a sixth criteria and definitively including all community schools within the final ranking.

The Ohio Department of Education (ODE) recently released a “preview” of these rankings for schools and districts, a simplistic ranking based solely on Ohio’s standardized test results as reported in Performance Index Scores. This release was so underwhelming that even The Columbus Dispatch broke out the sarcasm when covering ODE’s announcement:

The Ohio Department of Education has issued an official ranking of schools and districts by test-score performance. Never mind that this information already was available and easily sortable, if you wanted to rank them on your own.

(Columbus Dispatch, 11/10/2011)

We aren’t satisfied either, so we’ve been working on trying to assemble the rankings as described in the budget bill (and listed below). It hasn’t been easy, and we have learned more through the process of assembling the data than we did by actually constructing the final list.

This is the first of a series of posts in which we will dissect the Governor’s ranking system, explain why it is flawed in ways that can best be described as naive, at best, or at worst, vengeful. Finally, in the end, we will publish our comprehensive list of school districts, a list full of defects and exceptions due to the uninformed and unrealistic expectations laid upon ODE by Governor Kasich and his devotees.

The six criteria were adopted in the final bill as follows [bold added for emphasis]:

1. Performance index score for each school district, community school, and STEM school and for each separate building of a district, community school, or STEM school. For districts, schools, or buildings to which the performance index score does not apply, the superintendent of public instruction shall develop another measure of student academic performance and use that measure to include those buildings in the ranking so that all districts, schools, and buildings may be reliably compared to each other.
2. Student performance growth from year to year, using the value-added progress dimension, if applicable, and other measures of student performance growth designated by the superintendent of public instruction for subjects and grades not covered by the value-added progress dimension;
3. Performance measures required for career-technical education under 20 U.S.C. 2323, if applicable. If a school district is a “VEPD” or “lead district” as those terms are defined in section 3317.023 of the Revised Code, the district’s ranking shall be based on the performance of career-technical students from that district and all other districts served by that district, and such fact, including the identity of the other districts served by that district, shall be noted on the report required by division (B) of this section.
4. Current operating expenditures per pupil;
5. Of total current operating expenditures, percentage spent for classroom instruction as determined under standards adopted by the state board of education;
6. Performance of, and opportunities provided to, students identified as gifted using value-added progress dimensions, if applicable, and other relevant measures as designated by the superintendent of public instruction.

The department shall rank each district, community school, and STEM school annually in accordance with the system developed under this section.

One of the most important things to highlight before going any further is the amount of additional work that is placed on the Ohio Department of Education in this tiny section of HB153. Numbers 1, 2, and 6 all require the department to develop (technically says designate, but items don’t exist at this time) measures to cover gaps in this legislation. That means that half of the requirements are incomplete. Of greater significance is the fact that the Ohio Department of Education has also been tasked with creating other new processes out of thin air, most significantly a performance-based pay system and a new evaluation process for teachers and principals (neither of which have any real-world examples to pull from). But wait, there’s more! Earlier in November it was announced that the state is taking over sponsorship of eight charter schools whose sponsor was revoked because of changes elsewhere in Kasich’s budget (ODE is permitted to manage up to 20).

But if you remember nothing else about this explanation of ODE’s additional workload, know this: Just two days after he was hired, State Superintendent Stan Heffner was forced to layoff 26 employees (with more pending) because the same budget that assigned them additional work also cut operating funding for the ODE by $6.3 million, or 12.6 percent, over the next two years. Again, we’re not in the Stan Heffner Fan Club, but he explicitly advocated against these cuts in his Senate Finance Committee testimony:

However, we have some concern about ODE’s ability to assume all of the new responsibilities included in the budget bill, many of which I have already described, in addition to maintaining our ongoing regulatory and administrative responsibilities, given the cuts to our operations included in the House version of the bill. We understand that every agency must take its fair share of reductions to address the budget shortfall, but respectfully ask that administrative funding be restored to the levels proposed by the Executive version of the budget.

The combination of those details positively informs us that the Governor’s supposed implementation of high-quality education reforms is no more than lip-service (and ignorance). How can we expect high-quality reforms from an agency that is forced to cut staffing while also being tasked with crafting innovative new systems? We can’t, plain and simple.

If you aren’t pessimistic about the viability of the rankings yet, we’ve got more.

Let’s tackle these six criteria first.

#1 – Performance Index Scores. This is the easiest to obtain and provided the basis for the ODE-released scores. We said easy, not accurate or fair. Since these scores are based on standardized test results, only school districts with tested students with have a score (grades 3-8, 10). ODE’s simply opted to solve this issue by exempting the 65 school districts with a PI score from the released rankings. The legislature’s solution to this gap in data was to direct the Ohio Department of Education to create a comparable measure (within the next year). John Kasich somehow believes that ODE can whip up a measure of student evaluation that is of equal validity to the Ohio testing system that has been in use for 20 years. Does Kasich have the first clue about student evaluation or testing? Validity? Reliability? Drawing their naiveté out even further, we can point out that entire research papers have been devoted to the notion that standardized testing has many biases, with poverty being significant. We won’t dwell on this topic much in this post, but the chart below compares school districts’ PI scores with the percentage of economically disadvantaged students (often described as poverty level). Using the trend-line as a guide, it’s easy to see that in Ohio as poverty increases, the test results and connected PI scores decrease measurably. [Click image to enlarge.]

We could post graph after graph displaying this connection, but again, entire papers have been dedicated to proving this negative relationship between test scores and poverty. We point it out because it also demonstrates a bias with this first measurement against districts serving low-income students. Most significantly in Ohio, this negatively impacts the large urban school districts and, since they exist predominantly in low-income districts, Ohio’s charter schools. While we are certainly on record as being opposed to the charter school movement in Ohio, the effect of this criterion on charters supports the notion of Kasich’s ignorance surrounding these rankings. Kasich and his loyal followers (and his financial backers) are huge charter school advocates and have pushed for the expansion of charter schools across the state. At the same time, the inclusion of PI scores as a key statistic in ranking his friends’ schools puts a roadblock in their path to success.

This one single measure required in the rankings illuminates the incompetence of Kasich et al on many different levels.

That leaves only five more criteria that they’ve screwed up…..

We’ll expose the failures of more of the school ranking criteria in the next post in this series. If you have questions, please use the comments section below and we’ll try to address them throughout the posts.