Assessments · Method

How we measure, and what we don't claim

The same method in every region, written twice: once for anyone, once for the reviewer who wants to check our work.

In plain language

The question. When a manufacturer hires a recent graduate for an entry-level job, how close does that graduate come to what the employer actually needs?

Who sets the bar. The employers who do the hiring. In interviews, each employer rates each skill on a three-level scale: aware of it, competent at it, proficient at it. Their answers, taken together, are the region's expectation for that skill.

How programs are assessed. We go into the high school shops, talk with the instructors, and look at the equipment. For each skill we record what students can actually do by graduation, on the same three-level scale.

How the two are compared. For every skill, at every program, we ask whether the program's level meets the employers' bar. The share of skills where it does is that program's alignment. The region's alignment is the picture across all its programs.

What the percentage means. An alignment of 50 percent means students meet the employers' bar on half of the skills employers expect. It says nothing about how many students enroll or complete, or how good the teaching is. It measures what is offered and demonstrable, against what is expected.

What comes with it. Every gap is connected to the stakeholders and funding able to close it, and every employer's offer to plug in is recorded so the coalition can act on it.

[This section is drafted from the July 2026 one-pager. Before it goes live it must be checked line by line against the methodology document (data/ode_pos_deep_dive/ODE_POS_methodology.md) and the RNA 2.0 brief, and any difference resolved in favor of those documents.]

For the reviewer

[The rigorous version, per the RNA standard: the federal taxonomies used (SOC, O*NET, CIP, NAICS), the deterministic core that produces the citable result, how any judgment or model step is flagged and human-reviewed, precision, recall, and Cohen's kappa against a labeled truth set where available, the enumerated limitations, and the sensitivity notes for every weighting choice. To be drafted from the methodology document; every claim traceable to it.]

Limitations we state every time

  • An assessment measures what programs offer and what instructors say students can do. It does not measure enrollment, completion, or program quality.
  • Employer expectations come from the employers interviewed, not from every employer in the region.
  • Preliminary figures are labeled preliminary until the report is final.
  • Two regions assessed in different years are compared on the same scale, but not on the same day; regional economies move.

Planning an assessment, a coalition, or a training year?

Tell us the region and what you're trying to move. We answer within two business days.

Talk to us about your region