School Test Scores That Don't Add Up - and that surprises you?

March 9, 2014 -      I wrote on this same topic over the years. - Please read my write-up from 2011 below this current update.

I read in my local newspaper today about an article that describes the results from the state MEAP results for student testing.

http://thenewsherald.com/articles/2014/03/09/news/doc531a1a4112bde098112937.txt

http://thenewsherald.com/articles/2014/03/09/news/doc5319e1eed57d2837907197.txt

My own District - http://thenewsherald.com/articles/2014/03/06/news/doc53187d744b0cb739540056.txt

What you will find are words like "Trending, Results, Review, Average, Analyze, Compare"... These are Result Measurements and Effective Statements that continue to provide non-systemic answers to the continuing saga of not knowing how to fix or provide process improvement type solutions. Process (Result) and Efficient (Effectiveness) measurements must be established so the Voice of the Process can be identified and improved on an on-going basis.

The key word in my last statement is Process. Never do I read where students (Key input in the education Process) are tracked through out (Quality People call this Throughput) the education system. The educational leaders making comments/statements want to use the end result data to describe the problem and that is just plain wrong and no one cares because they hope the next test results will be better. Many of the folks making these comments or reviewing the data were not involved in the previous data collections and now try to figure out the data from a current perspective and rely on the next proposed educational solutions (Common Core) to solve the same existing problem.

I will know progress is being made when I hear/read those process owners/leaders of their educational system(s) start providing data and solutions prior to the reports coming out. The Test Results should for the most part already be known if they have the correct process/efficiency measures in place. It's just like the auto companies of old that used to us how well they did after the car was built. Quality Control was at the end of the process. I believe in many cases Quality Control for education is still at the end of the process being driven by after the fact result figures, data and information.

We must track the data through the process from beginning to end using the students (Good random sample size) as one of the key measuring points through the process. Ask yourself these questions:

Do your data gathers track your school data by Grade Level like the results or do they track individual students starting in Kindergarten and follow them through high school?  Do your Data Gathers identify students in trouble and track these students for improvement? Do your Data Gathers share and educate the teachers in the process from year to year? Do your Data Gathers break this data down for root cause analysis by including teachers and parents in the process?



Click on Pic to Enlarge

Schools need Fact Based Data Information and Specialists to create and implement solid improvement plans. Until then I will continue to update these posts every few years as I continue to be frustrated.


March 6, 2011,

Here we go again. Another article about the shambles of our education system; particularly our test scores.

We what the heck did you expect. All we do is Tamper and establish non value added solutions and after the fact assessments using results driven short term data and distorted test results.

The following is an exchange of some simple questions that go on every morning when results driven staff meetings take place (regardless of industry).

How are we doing? Uh, not so good How can we do better? Uh, we have some ideas Are we achieving the results that we want? Uh, we are not sure what you want or mean
These are such simple questions. Uh, well, Uh,
Why is it so difficult to get answers? Uh, the people won't do what we want, but we are going to give them some ultimatums
OK, that sounds good, let me know the results of all the changes you implement and give me the names of the people responsible. Uh, will do (now imagine a bunch a people running to the bathroom to .... their drawers and take there daily dose of Pepto Bismal - HA!)

Let me provide some wisdom provided by Dr. W. Edwards Deming on this subject – “Dr. Deming spoke of 3 ways to improve (Does not imply "good") the figures: distort the data, distort the system and improve the system. Improving the system is the most difficult.”

Following is a prime example of distorting the data:

In today’s front page of my paper (Detroit Free Press 3/6/2011) I read:


When test scores don't add up: 32 metro Detroit schools show improvements too good to be true
Each year, millions of children in Michigan and across the nation take state standardized tests that impact everything from a school's reputation to how teachers will be evaluated to whether schools will even survive.
The pressures to perform, experts say, tempt some school administrators and teachers to cheat.

The Free Press, as part of a nationwide investigation with USA TODAY and other partners, analyzed millions of test score results and found that 34 schools across Michigan -- 32 of them in metro Detroit -- showed test score gains over a one-year period that experts say are statistically improbable. More broadly, the analysis found 304 schools in six states and the District of Columbia that had test scores so improbable, they should be investigated. Besides Michigan, the states were Arizona, Colorado, California, Florida and Ohio.


Rest of story - http://www.freep.com/article/20110306/NEWS06/103060602/When-test-scores-don-t-add-up-32-metro-Detroit-schools-show-improvements-too-good-true?odyssey=tabtopnewsimgFRONTPAGE


Here is the comment that got me the most “Huge gains by some metro Detroit schools should be red flags of tampering, experts say.

Want to know about "Tampering" - enter the word in the search box on the right hand side of this blog.

Are you kidding me? Experts? Are you kidding me? This is the same crap that goes on in the business world everyday and I have written about tampering so many times bases on the teachings of Dr. Deming. Experts? Take the time to understand what it means to improve a system (any system).


“Distort the data, distort the system and improve the system. Improving the system is the most difficult.”


Educational Leaders (national education czar, state educational directors, superintendents, and principals) are responsible for the system. STUDENTS, administrative staff, teachers and support staff work in the educational system and while they are responsible for their own behavior, they cannot be held accountable for the results of the system. For example, last month, many students crammed for mid-term exams. Teachers hold students accountable for all learning and they "measure" their learning by exams! Cramming for exams is how students attempt to distort the system (i.e., the learning system) in order to get demanded results.


Other profound statements about “Distort the data, distort the system and improve the system. Improving the system is the most difficult.”


The following is an excerpt from Chapter 4 of The New Economics, second edition by W. Edwards Deming:

A teacher, not wishing to penalize anyone unjustly, will pass a pupil that is barely below the requirement for a passing grade.

Fear invites wrong figures. Bearers of bad news fare badly. To keep his job, anyone may present to his boss only good news.

A committee appointed by the President (regardless of industry i.e., Superintendent) of a company will report what the President wishes to hear. Would they dare report otherwise?

An individual may inadvertently seek to cast a halo about himself. He may report to an interviewer in a study of readership that he reads the New York Times, when actually this morning he bought and read a tabloid.

Statistical calculations and predictions based on warped figures may lead to confusion, frustration, and wrong decisions.

Accounting-based measures of performance drive employees to achieve targets of sales, revenue, and costs, by manipulation of processes, and by flattery or delusive promises to cajole a customer into purchase of what he does not need (adapted from the book by H. Thomas Johnson, Relevance Regained, The Free Press, 1992).

A leader of transformation, and managers involved, need to learn the psychology of individuals, the psychology of a group, the psychology of society, and the psychology of change.

Some understanding of variation, including appreciation of a stable system, and some understanding of special causes and common causes of variation, are essential for management of a system, including management of people.

Want more examples that I have found read my blogs about Dr. Deming and Education. Just click on the labels below or on the left side bar of the web site.

Do yourself a favor - Google or Yahoo or Bing or whatever and learn about the teachings of Dr. Deming. Maybe then we can start to transform our educational system. Not Change but Transform!