Situation

The Port of Portland created a tier of operational management at Portland International Airport: the Airport Operations Supervisor. Was the training for these new employees effective? Where the new employees fulfilling stakeholder expectations?

Solution

Steaming Kettle performed a Kirkpartick Level Three evaluation of the Airport Operations Supervisor Training. Four evaluation components were used to reveal the affect of training on on-the-job behavior:

• A survey of Airport Operations Supervisors who completed the training program 
• One-on-one interviews with every active Airport Operations Supervisor 
• A survey of airport workers (from inside and outside the Port of Portland) who observed Airport Operations Supervisors on the job, and 
• A discussion forum in which survey participants gave more detailed feedback in person.

Using twenty-eight objectives created during training development, performance criteria were created and reviewed with managment. Evaluation materials included elements intended to assess different training modes, including slide presentations, certification exams, tours, tabletops, and on-the-job training.

Result

Stakeholders were extremely responsive to surveys and discussions: the effort achieved a 94% response rate to surveys.

Steaming Kettle's Kirkpartick Level Three evaluation revealed a highly successful training program, meaning that training had a direct impact on positive job performance.

One metric of note is that airfield management observed a significant reduction in calls relating to inconsistencies that required intervention: the volume of such calls has dropped from approximately one per week to approximately one per month.

The two improvement categories that received the most attention were:

• The type and depth of on-the-job training 
• Refining and training the role of Airport Operations Supervisors in PDX operations