Evaluation

Applying assessment and evaluation in instructional design practice.

Evaluation Plan: Online Leadership Development Program

Course: EDIT 6170E – Introduction to Instructional Systems Design
Instructor: Dr. Jill Stefaniak
Industry: Corporate Learning, Leadership Development, Online Training
Tools: Logic models, stakeholder analysis, mixed-methods research, system evaluations, and evaluation models (e.g., Kirkpatrick Model).
Responsibilities: Evaluation planning, stakeholder analysis, logic modeling, mixed-methods design, formative/summative assessment, confirmative evaluation, leadership training and organizational needs analysis, evaluation instrument design, and systems-based alignment.

This artifact presents an evaluation plan developed for an online leadership development program supporting Director-level employees transitioning into Vice President roles. The evaluation framework incorporated logic modeling, stakeholder analysis, and mixed-methods strategies to assess engagement, leadership growth, skill transfer, and organizational impact. Formative, summative, and confirmative evaluation approaches were used across training modules and workplace contexts to measure both learning outcomes and performance application. Overall, the project demonstrates the design of a comprehensive evaluation system aligned with organizational goals and leadership development outcomes in a corporate online learning environment.

Project Context & Highlights

Problem

Director-level employees preparing to transition into Vice President roles required stronger leadership, coaching, strategic thinking, and team management skills. After the program shifted from in-person delivery to an online format during the COVID-19 pandemic, stakeholders needed a structured way to evaluate learner engagement, leadership growth, skill transfer, and long-term organizational impact within a virtual learning environment.

Solution

A comprehensive, human-centered evaluation plan was developed using a logic model and mixed-methods framework to align instructional activities, learner experiences, and organizational outcomes. The evaluation integrated formative, summative, and confirmative assessment strategies to measure leadership development across six core modules, including self-leadership, coaching, team leadership, strategic decision-making, and leadership branding.

Highlights

  • Use of a logic model to connect inputs, activities, outputs, short-term outcomes, and long-term organizational goals
  • Embedding formative evaluation checkpoints throughout the online learning experience
  • Inclusion of confirmative evaluation methods to assess sustained leadership application over time
  • Alignment of evaluation measures with six leadership development modules and real-world leadership responsibilities
Design Decisions and Reflection

Process Overview

This artifact was developed as part of a collaborative evaluation project centered around a real-world client and authentic organizational training materials. The report was constructed using client-provided artifacts, including PowerPoint presentations, facilitator materials, workbooks, and background information about the organization and its leadership development program. These materials were analyzed to design and conduct a comprehensive evaluation process.

The evaluation plan was developed to help stakeholders assess the effectiveness of the online leadership training program by examining several key areas: the instructional design and structure of the training materials, learner engagement and participation within the online environment, the real-world application of leadership skills in workplace settings, and the alignment of the training with the company’s broader leadership development goals and organizational needs.

The evaluation process focused on multiple areas of leadership training effectiveness, including learner engagement, knowledge retention, skill application, peer collaboration, and long-term leadership growth. To measure these areas, the team implemented formative, summative, and confirmative evaluation methods supported by mixed-methods data collection. Surveys, interviews, peer feedback, observations, pre- and post-assessments, and follow-up sessions were used to gather both qualitative and quantitative data. Long-term evaluation measures included manager feedback, leadership performance, and continued use of acquired skills after the completion of training.

We conducted a stakeholder analysis to identify individuals who may influence or impact the program and to gauge their level of involvement. A logic model was developed to outline organizational needs and goals, ensuring alignment between program objectives, activities, and evaluation methods. Program inputs, resources, activities, and outcomes were analyzed for alignment with the company’s goals and objectives.

An evaluation analysis was conducted to identify learner needs, organizational goals, performance gaps, and contextual factors before and during the design process. The evaluation was used to draw inferences about the overall effectiveness of the program. From there, evaluation instruments composed of mixed-methods data were developed.

Justification for Inclusion

This artifact is included as it demonstrates the design of evaluation systems rather than only the evaluation of systems. It provides evidence of the development of formative, summative, and confirmative evaluation tools, as well as an evaluation plan designed to assess the quality and performance outcomes of a given project. The artifact also demonstrates how evaluation and assessment can inform effectiveness. It situates evaluation within context to demonstrate alignment between organizational goals, needs, and the logic model.

Value and Self Assessment

This project pushed me outside of my typical comfort zone and into a space where I had to think beyond instructional design and consider full system-level impact. I found value in shifting my perspective to examine how inputs, processes, and outputs connect within a larger organizational system.

Working through this evaluation also required me to make sense of limited and sometimes ambiguous information, which strengthened my ability to form structured inferences and design coherent evaluation systems. I enjoyed the challenge of translating complex organizational needs into a clear evaluation framework.

Overall, this experience expanded my thinking from designing learning experiences to designing evaluation systems that consider multiple layers of impact, including learner outcomes and organizational goals. It strengthened my systems thinking and helped me see myself more broadly as a designer working across performance, learning, and organizational effectiveness.

Facilitator & Participant Guides for In-person Design Sprint

Course: EDIT 6170E – Introduction to Instructional Systems Design
Instructor: Dr. Jill Stefaniak
Industry: Professional Development, Adult Learning, Instructional Design
Tools: Canva, PowerPoint, collaborative learning strategies, human-centered design activities, facilitation planning documents, participant support resources
Responsibilities: Facilitator guide development, participant guide creation, instructional sequencing, collaborative activity design, design sprint facilitation planning, human-centered learning integration, workshop structure design, learner engagement strategy development, and professional learning support materials.

Instructional design artifact demonstrating the development of facilitator and participant guides grounded in adult learning theory, social learning, and constructivist design principles for an in-person design sprint professional development. These guides translate adult learning theory into structured, facilitative learning experiences where educators engage in empathy-building, AI-supported reflection, and collaborative design thinking activities during an in-person design sprint.

Project Context & Highlights

The Problem

Professional development often lacks experiential, design-based structures that provide teachers with quick, effective methods for designing learner-centered experiences using an engineering and design-thinking approach.

The Learning Solution

These facilitator and participant guides provide a structured, step-by-step process for in-person professional development that helps teachers quickly design and implement learner-centered experiences using empathy mapping, collaboration, and design thinking.

Highlights

  • Evaluation is structured through multiple perspectives (learner and facilitator), using ongoing formative assessment loops to monitor understanding and adjust instruction in real time
  • Assessment integrates both informal and formal methods to capture learner engagement, understanding, and performance during collaborative and applied learning experiences
  • Performance-based tasks serve as evidence of learning, allowing facilitators to evaluate application of skills in authentic, design-based contexts
  • Evaluation of engagement is aligned with Kirkpatrick’s model, focusing on learner reaction, participation, and active involvement in the learning process
Design Decisions and Reflection

Process Overview

This artifact demonstrates Theme 4, Evaluation, through the intentional integration of formative and summative evaluation strategies embedded throughout both the facilitator and participant guides. The guides were developed as part of a human-centered design sprint experience intended to support educators in implementing empathy-driven and inquiry-based learning experiences within science education. The learning experience was designed around collaborative problem-solving, reflective discussion, rapid prototyping, and iterative feedback processes that allowed participants to continuously evaluate both their learning and the effectiveness of proposed solutions. The artifact demonstrates the ability to evaluate the success of a learning experience and/or product by embedding structured opportunities for reflection, feedback, testing, discussion, and revision throughout the instructional process.

Justification for Inclusion

This artifact was included because it strongly aligns with several aspects of Theme 4: Evaluation. The guides demonstrate the ongoing nature of conducting formative evaluations through ongoing reflection prompts, facilitator questioning strategies, collaborative discussions, peer feedback, and observation opportunities embedded across all phases of the design sprint. Rather than treating evaluation as a single event at the conclusion of instruction, the guides position assessment and evaluation as ongoing and diversified processes that occur continuously throughout the learning experience. Participants regularly engage in reflective questioning, prototype testing, ideation discussions, empathy activities, and peer critique sessions that provide facilitators with continuous insight into learner understanding, engagement, collaboration, and application of concepts.

Additionally, the artifact demonstrates the development of summative evaluative tools through the inclusion of reflective discussion prompts, end-of-session surveys, voting activities, and final reflection opportunities that allow facilitators to assess participant perceptions of the experience and the effectiveness of the instructional design. The assessment and evaluation processes within both documents also provide windows into the perspectives of both the learner and the facilitator. The participant guide supports learner self-reflection, collaboration, and feedback, while the facilitator guide supports instructional decision-making by providing structured prompts, facilitation cues, and questioning strategies designed to help facilitators monitor participant understanding and engagement throughout the sprint.

Another important feature of the artifact is the use of facilitator icons, prompts, and structured discussion cues that guide facilitators in recognizing the types of responses, evidence of understanding, and reflective dialogue they should listen for while participants engage in activities. These supports help facilitators conduct ongoing formative evaluations in real time while adapting instruction, probing for deeper understanding, and supporting collaborative meaning-making. The guides also demonstrate the ability to conduct usability testing and evaluate instructional interventions through prototype testing sessions, peer review activities, dot-voting exercises, and iterative refinement opportunities. Participants are encouraged to evaluate the feasibility, desirability, sustainability, and effectiveness of proposed solutions while reflecting on challenges and opportunities for improvement.

Value and Self Assessment

The value of this artifact lies in its emphasis on evaluation as an embedded and reflective instructional process rather than an isolated assessment activity. The artifact models how evaluation can be integrated naturally into experiential and constructivist learning environments while supporting empathy, collaboration, creativity, and authentic problem-solving. One strength of this project is the intentional integration of multiple perspectives, including facilitator observations, learner reflections, peer feedback, and collaborative evaluation processes. In reflecting on this work, I recognize opportunities for future growth, including expanding usability testing measures, collecting longitudinal implementation data, and incorporating additional quantitative evaluative measures to further assess the long-term impact of the learning experience.