The New York Times Magazine article What it Will Really Take to Close the Education Gap points up a common challenge in measurement and benchmarking: we often set our benchmarks too high. Doing so can undermine credibility and ultimately support for a particular strategy. Take the case of No Child Left Behind, the ambitious law that requires states to achieve 100% proficiency in reading and math for all students "not later than 12 years after the end of the 2001-2002 school year."

The NCLB legislation does many things right. First, NCLB clearly lays out the outcomes, or ultimate objectives, that the legislation desires to achieve and builds in a requirement for accountability. Second, NCLB allows each state (i.e. funding recipient) to determine intermediate benchmarks for progress while setting parameters for how those benchmarks should be set. And third, the legislation allows each state to create its own strategy for improving academic performance. Finally, the legislation holds states accountable for performance, by withholding funding and requiring a plan for corrective action. There are detailed rules promulgated by the U.S. Department of Education on how the measurement regime is to be operationalized.

The biggest challenge with NCLB may be its amibition. NCLB states that all students should be proficient in reading and math within 12 years. There are a number of problems with this benchmark.

First, the benchmark is extremely ambitious, if not unattainable. Setting a benchmark too high can lower motivations of the project participants, undermine support for the project itself and provide perverse incentives to skew behavior. The key question for NCLB is: is 100% proficiency a reasonable goal? Setting the benchmark correctly requires more than just pulling a number out of the sky. One must consider a number of important variables in setting the right benchmark.

1. Are sufficient resources being spent? Is approximately $20Bn/year for NCLB enough? Has someone analyzed the cost of progress and projected that this level of spending can reasonably be required to generate the desired result?

2. How much time is reasonable to achieve the benchmark? Can states get to 100% proficiency within 12 years? Is that too long? Where did that number come from?

3. What's the baseline? Getting to the benchmark depends on the distance from the baseline - i.e. where performance is currently. State schools are at vastly different levels of proficiency, with different challenges. Therefore, setting different benchmarks for each may make more sense than an overall blanket benchmark.

4. Are the strategies proven effective? The level of confidence in reaching a benchmark is much higher if the program is following a proven strategy versus experimenting with new approaches. There are many cases when new approaches are required, but experimentation may extend the timeframe for success. NCLB does not prescribe a particular strategy, which means that there will be many different approaches undertaken to raising test scores. It may take time for states to determine which strategy works best.

5. Have the recipients agreed to the benchmark? Setting a benchmark is a negotiation - between the performers (i.e. recipients of funding) and the underwriters. To be effective, and enforceable, one needs buy-in upfront. It is unclear how much input governors or state education officials had to the timeframe, resource requirements or benchmarks.

6. Does the benchmark measure the right thing? Does achieving the benchmark answer the "so what" test? Under NCLB, "success" is measured by students reaching proficiency in math/science and reading. Is 100% "proficiency" at these subjects a valuable enough outcome? Would a 75% college-going rate (which presumes proficiency) be a better value? Would 50% year-to-year increase in sub-standard student test scores be more reasonable?

No doubt, benchmarking and accountability are good things, and NCLB does a good job at both. Now we need to "benchmark" the success of our own benchmarking, ensuring that we measure the right things, in the right ways, so that the practice itself is successful.