Evaluation

Applying assessment and evaluation in instructional design practice.

Evaluation Plan: Online Leadership Development Program

Course: EDIT 6170E – Introduction to Instructional Systems Design
Instructor: Dr. Jill Stefaniak
Industry: Corporate Learning, Leadership Development, Online Training
Tools: Logic models, surveys, interviews, stakeholder analysis frameworks, mixed-methods research strategies, evaluation planning tools, PowerPoint, collaborative analysis documents
Responsibilities: Evaluation planning, stakeholder analysis, logic model development, mixed-methods evaluation design, formative and summative assessment development, confirmative evaluation planning, leadership training analysis, organizational needs analysis, evaluation instrument design, and systems-based evaluation alignment
Tools: Logic models, surveys, interviews, stakeholder analysis frameworks, mixed-methods research strategies, evaluation planning tools, PowerPoint, collaborative analysis documents

This artifact presents the development of a human-centered evaluation plan designed for an online leadership development program supporting Director-level employees transitioning into Vice President roles. The evaluation framework integrated logic modeling, stakeholder analysis, and mixed-methods research strategies to assess learner engagement, leadership growth, skill transfer, and long-term organizational impact. Formative, summative, and confirmative evaluation methods were used to measure leadership development across multiple training modules and workplace contexts. Overall, the project demonstrates the design of evaluation systems that align instructional experiences, organizational goals, and leadership performance outcomes within a corporate online learning environment.

Problem

Director-level employees preparing to transition into Vice President roles required stronger leadership, coaching, strategic thinking, and team management skills. After the program shifted from in-person delivery to an online format during the COVID-19 pandemic, stakeholders needed a structured way to evaluate learner engagement, leadership growth, skill transfer, and long-term organizational impact within a virtual learning environment.

Solution

A comprehensive, human-centered evaluation plan was developed using a logic model and mixed-methods framework to align instructional activities, learner experiences, and organizational outcomes. The evaluation integrated formative, summative, and confirmative assessment strategies to measure leadership development across six core modules, including self-leadership, coaching, team leadership, strategic decision-making, and leadership branding.

Design Decisions Reflection
  • Use of a logic model to connect inputs, activities, outputs, short-term outcomes, and long-term organizational goals
  • Integration of mixed-methods data collection including surveys, interviews, reflections, and stakeholder feedback
  • Embedding formative evaluation checkpoints throughout the online learning experience
  • Inclusion of confirmative evaluation methods to assess sustained leadership application over time
  • Alignment of evaluation measures with six leadership development modules and real-world leadership responsibilities
  • Positioning learners, facilitators, and stakeholders as active contributors to continuous program improvement
Design Decisions and Reflection

Process Overview

This artifact was developed as part of a collaborative evaluation project centered around a real-world client and authentic organizational training materials. The report was constructed using client-provided artifacts, including PowerPoint presentations, facilitator materials, workbooks, and background information about the organization and its leadership development program. These materials were analyzed to design and conduct a comprehensive evaluation process.

The evaluation plan was developed to help stakeholders assess the effectiveness of the online leadership training program by examining several key areas: the instructional design and structure of the training materials, learner engagement and participation within the online environment, the real-world application of leadership skills in workplace settings, and the alignment of the training with the company’s broader leadership development goals and organizational needs.

p>The evaluation process focused on multiple areas of leadership training effectiveness, including learner engagement, knowledge retention, skill application, peer collaboration, and long-term leadership growth. To measure these areas, the team implemented formative, summative, and confirmative evaluation methods supported by mixed-methods data collection. Surveys, interviews, peer feedback, observations, pre- and post-assessments, and follow-up sessions were used to gather both qualitative and quantitative data. Long-term evaluation measures included manager feedback, leadership performance, and continued use of acquired skills after the completion of training.

We conducted a stakeholder analysis to identify individuals who may influence or impact the program and to gauge their level of involvement. A logic model was developed to outline organizational needs and goals, ensuring alignment between program objectives, activities, and evaluation methods. Program inputs, resources, activities, and outcomes were analyzed for alignment with the company’s goals and objectives.

An evaluation analysis was conducted to identify learner needs, organizational goals, performance gaps, and contextual factors before and during the design process. The evaluation was used to draw inferences about the overall effectiveness of the program. From there, evaluation instruments composed of mixed-methods data were developed.

Justification for Inclusion

This artifact is included as it demonstrates the design of evaluation systems rather than only the evaluation of systems. It provides evidence of the development of formative, summative, and confirmative evaluation tools, as well as an evaluation plan designed to assess the quality and performance outcomes of a given project. The artifact also demonstrates how evaluation and assessment can inform effectiveness. It situates evaluation within context to demonstrate alignment between organizational goals, needs, and the logic model.

Value and Self Assessment

The value of this artifact is that it demonstrates the ability to construct and design an evaluation plan based on inferential analysis of limited client-provided materials. It further shows the ability to conduct the necessary analysis to develop an evaluation framework that meets the needs and goals of the client. This also demonstrates not only my ability to think systemically, but also to translate systems thinking into evaluation design.

Facilitator & Participant Guides for In-person Design Sprint

Course: EDIT 6170E – Introduction to Instructional Systems Design
Instructor: Dr. Jill Stefaniak
Industry: Professional Development, Adult Learning, Instructional Design
Responsibilities: Facilitator guide development, participant guide creation, instructional sequencing, collaborative activity design, design sprint facilitation planning, human-centered learning integration, workshop structure design, learner engagement strategy development, and professional learning support materials
Tools: Canva, PowerPoint, collaborative learning strategies, human-centered design activities, facilitation planning documents, participant support resources

Instructional design artifact demonstrating the development of facilitator and participant guides grounded in adult learning theory, social learning, and constructivist design principles for an in-person design sprint professional development. These guides translate adult learning theory into structured, facilitative learning experiences where educators engage in empathy-building, AI-supported reflection, and collaborative design thinking activities during an in-person design sprint.

Facilitator Guide Preview Participant Guide Preview
The Problem

Professional development often lacks experiential structures that build empathy and model instructional strategies for teachers.

The Learning Solution

These guides support facilitators in delivering inquiry-based learning experiences that integrate empathy mapping, collaboration, and design thinking.

Why This Approach Works
  • Adult learning theory emphasizes relevance, experience, reflection, and application
  • Social learning is strengthened through collaboration and modeling
  • Constructivist design supports authentic meaning-making through practice
Design Decisions Reflection

Overview

This artifact demonstrates Theme 4, Evaluation, through the intentional integration of formative and summative evaluation strategies embedded throughout both the facilitator and participant guides. The guides were developed as part of a human-centered design sprint experience intended to support educators in implementing empathy-driven and inquiry-based learning experiences within science education. The learning experience was designed around collaborative problem-solving, reflective discussion, rapid prototyping, and iterative feedback processes that allowed participants to continuously evaluate both their learning and the effectiveness of proposed solutions. The artifact demonstrates the ability to evaluate the success of a learning experience and/or product by embedding structured opportunities for reflection, feedback, testing, discussion, and revision throughout the instructional process.

Justification for Inclusion

This artifact was included because it strongly aligns with several aspects of Theme 4: Evaluation. The guides demonstrate ongoing nature of conducting formative evaluations through ongoing reflection prompts, facilitator questioning strategies, collaborative discussions, peer feedback, and observation opportunities embedded across all phases of the design sprint. Rather than treating evaluation as a single event at the conclusion of instruction, the guides position assessment and evaluation as ongoing and diversified processes that occur continuously throughout the learning experience. Participants regularly engage in reflective questioning, prototype testing, ideation discussions, empathy activities, and peer critique sessions that provide facilitators with continuous insight into learner understanding, engagement, collaboration, and application of concepts.

Additionally, the artifact demonstrates the development of summative evaluative tools through the inclusion of reflective discussion prompts, end-of-session surveys, voting activities, and final reflection opportunities that allow facilitators to assess participant perceptions of the experience and the effectiveness of the instructional design. The assessment and evaluation processes within both documents also provide windows into the perspectives of both the learner and the facilitator. The participant guide supports learner self-reflection, collaboration, and feedback, while the facilitator guide supports instructional decision-making by providing structured prompts, facilitation cues, and questioning strategies designed to help facilitators monitor participant understanding and engagement throughout the sprint.

Another important feature of the artifact is the use of facilitator icons, prompts, and structured discussion cues that guide facilitators in recognizing the types of responses, evidence of understanding, and reflective dialogue they should listen for while participants engage in activities. These supports help facilitators conduct ongoing formative evaluations in real time while adapting instruction, probing for deeper understanding, and supporting collaborative meaning-making. The guides also demonstrate the ability to conduct usability testing and evaluate instructional interventions through prototype testing sessions, peer review activities, dot-voting exercises, and iterative refinement opportunities. Participants are encouraged to evaluate the feasibility, desirability, sustainability, and effectiveness of proposed solutions while reflecting on challenges and opportunities for improvement.

Overview

The value of this artifact lies in its emphasis on evaluation as an embedded and reflective instructional process rather than an isolated assessment activity. The artifact models how evaluation can be integrated naturally into experiential and constructivist learning environments while supporting empathy, collaboration, creativity, and authentic problem-solving. One strength of this project is the intentional integration of multiple perspectives, including facilitator observations, learner reflections, peer feedback, and collaborative evaluation processes. In reflecting on this work, I recognize opportunities for future growth, including expanding usability testing measures, collecting longitudinal implementation data, and incorporating additional quantitative evaluative measures to further assess the long-term impact of the learning experience.