In the national conversation about United States’ education system and student academic performance, the U.S.’s standing on standardizing tests in core subjects relative to other industrialized countries seems to be a core evidence point highlighting our struggles.  But as with most measurement efforts, key performance indicators often raise more questions than they answer.  This is a good thing.  These questions shouldn’t only give us pause when considering the meaning of the performance itself, but also provide an opportunity to reflect on our definition of success.

• How many students are exempted from those figures due to non-participation in the education system due to inaccessibility, learning disability, etc.?  Is the education of ALL students a key priority?  Is it for those countries against which we would compare ourselves?
• How are the cross-country comparisons being made?  How can we be sure that proficiency is being measured comparably?  (A 2007 study by the American Institutes for Research suggests mapping international exams such as TIMSS to US exams such as NAEP)
• What role does cultural competency play in the development of a test which can accurately assess ability?  As one of the most culturally diverse nations, is the US at an inherent disadvantage?
• How predictive are standardized test scores of post-secondary success? After all, success on a standardized test is meaningless if it only means that you are skilled at taking standardized tests.

I find the last question most interesting.  A recent article in Time Magazine on “helicopter parenting” describes an example of the skills beyond those assessed in standardized tests.  In this case, it highlights the importance of creative problem solving for engineers, which historically has been developed through unstructured play:
 
“Dr. Stuart Brown, a psychiatrist and the founder of the National Institute for Play — who has a treehouse above his office — recalls in a recent book how managers at Caltech's Jet Propulsion Laboratory (JPL) noticed the younger engineers lacked problem-solving skills, though they had top grades and test scores. Realizing the older engineers had more play experience as kids — they'd taken apart clocks, built stereos, made models — JPL eventually incorporated questions about job applicants' play backgrounds into interviews.”
 
As we continue to look for ways to improve the nation’s education system, it is important that we keep in mind the ultimate goal of public education system, producing a capable, productive citizenry.  And in doing so, we should be using metrics along the way that indicate progress towards that end.  While standardized tests can be an important check-point and useful comparison to other schools, districts, states, and countries, let’s also realize their limitations.  Let’s be careful not to confuse a good “mid-term” grade with actually achieving our ultimate mission.