Analysis

By: James NicholsSharon Walsh and Muhammed Yaylaci.  Edited by Steve Covello

About Analysis

Imagine that you have been hired by the Acme Widget Corporation in response to a situation where, according to management, staff office workers cannot accurately process the hourly payroll tasks using the new Qlu-G Payroll Management tracking system. According to the management team, a training program is needed to straighten out this problem.

How does an instructional designer (ID) respond to this discovery? Start building a training program?

Yes, perhaps, but not quite yet. The ID needs to investigate the claim, gather relevant information about its causes and context, and study the subject matter. In fact, the problem might not have anything to do with the need for staff to attend training. Perhaps the problem has to do with management communication, a problem with organizational structure, or a limitation within the Qlu-G software?

The Analysis phase of the ADDIE model serves as the investigative prelude that informs, first of all, whether a program of instruction is needed, and then if so, what outcome should the program produce. In short, the Analysis stage orients the entire pathway of the instructional design process, which ultimately leads to producing a program of instruction to resolve the problem.

Analysis as inquiry

The ID uses a comprehensive set of questions and methods of inquiry to establish the foundation for subsequent phases in the ADDIE model, as seen in Table 1 below. The culmination of this work establishes the nature of the training or performance improvement problem.

Table 1. Critical questions in analysis.

Critical Questions Answers Report
Who is the focus of the instructional problem? The characteristics of the people (i.e. the learners) who will be engaged in instruction. Learner Analysis
What is the desired outcome of instruction? A statement about the desired result or outcome of the instruction in the form of a Learning Goal. Needs Assessment
When is the program of instruction needed by? How much time do learners have to complete it? The amount of time available for producing a program of instruction and the amount available for the learners to proceed through it. Context Analysis
Where must instruction take place? The context or environment the learners will need for the instruction to be effective or relevant. Context Analysis
Why is instruction needed? A statement that describes the necessity for the instruction. Needs Assessment
How will learning transfer? A statement about how learned skills, knowledge, and attitudes will transfer to the conditions under which the learning is relevant. Context Analysis
What is the nature of the task, knowledge, or performance? A detailed extraction of the physical and mental tasks, decisions, operations, expressions, choices, and alternatives associated with the subject matter. Task Analysis

Your goal in the Analysis stage is to produce the following sections:

Needs Assessment: A description of the problem and learning goal(s) to be achieved.

Learner Analysis: A description of the learners in terms relevant to instruction.

Context Assessment: A description of the conditions and timeframe under which the learned skills, knowledge, or attitude would be applicable.

Task Analysis: A an explication of a task, performance, or demonstration according to sequences, priorities, decisions, choices, and alternatives.

Communication methods for gathering information

The ID employs several forms of communication to gather information from to establish the nature of the training or performance improvement problem.

Interview: Conduct interviews across all levels of stakeholders to account for multiple perspectives on the nature of the problem and the subject matter.

Survey development: Issue surveys across all levels of stakeholders to account for specific questions on the nature of the problem and the subject matter.

Extant data: If there is data available that is relevant to the problem or subject matter, the ID must gather as much of it as feasible and relevant to identify performance trends.

Observation: The ID (and perhaps with an SME) can observe the conditions under which the problem is occurring. Focus can be on both the qualitative circumstances of performance and the means/methods by which the performance is conducted.

Focus groups: The ID convenes and facilitates groups of people to discuss relevant issues and elaborate upon their impressions.

Exams / tests: A pre-test can be administered to determine patterns or trends of current knowledge, skills, or attitudes in the population of the targeted group. If the instructional need involves training for performance on something completely new, this method might not be needed.

The rationale for employing a range of methods is driven by the possibility that the kind of information you receive from persons on the job may conflict with the perception of the performance by others in the organization. Your goal, as an ID, is to ascertain a perspective of the problem that reflects both the reality of the situation based on objective evidence and the perception of the problem based on individual subjective perspectives. They may or may not align, which lends itself to further discussion with stakeholders to determine the precise nature of the problem, and thus, the goal of the program of instruction.

Writing in plain language

As a communications expert, the value of your communication can only be measured by how effective it is received, which is often a factor of the audience to whom you are communicating. In each of the reports you will produce, you should keep in mind that the audience to whom you are presenting are not necessarily educators or experts in the field of instructional design. If you use jargon or unfamiliar terms, your audience may feel alienated, or worse, not even bother reading your reports.

Since most of your communication in your project plan will be in written form, consider the tonality of your writing. A report for a stakeholder audience should be less formal than a scholarly research paper, yet maintain professional writing standards. Using plain language in your reports will enable your stakeholders to better understand the complexities of the situation which will ultimately produce better results for your project.


Needs Assessment

As stated in Table 1, the Needs Assessment section describes the findings that answer the questions, “What is the desired outcome of instruction?” and “Why is instruction needed?”

The first part of Needs Assessment is to interrogate the nature of the problem as it has been presented to you. This is an important step because it is possible that the problem can be resolved without an expensive program of instruction. Or it is possible that the cause of the problem is not related to the knowledge, skills, or attitudes of the target employees or staff.

For example, suppose workers at a fast food restaurant have difficulty remembering the proper layers for each of the hamburgers on the menu. Rather than spending additional financial and staff resources in a training program, the workers may only need a job aid (a handout or guide) posted where they work to use as a visual reference. The problem, as it has been discovered, may be caused by part time workers who aren’t immersed in the process frequently enough to memorize the recipe for every item on the menu. They know how to do the work, but may just need a visual reminder. Additional training would not likely be as effective or feasible as providing an instructional guide.

The Needs Analysis is comprised of the following tasks.

I. Problem analysis

The ID needs to gather information from stakeholders that answers the following questions:

  • Who, or which parties, have determined that there is a problem? What is the basis for their position? The answers will determine whether the problem is limited to a particular person’s perspective or whether it is more substantial.
  • How is the problem perceived by other stakeholders? The answer will explain whether the situation is considered a problem by others. It might not be a problem beyond a limited set of conditions.
  • Under what conditions does the problem arise? The answer will explain whether the problem is occurring only under certain conditions or if it is more widespread.
  • Who or what is affected by this problem? The answer will identify the people or context that are experiencing the effects of the problem.
  • How severe or pervasive is the problem? The answer will describe the collateral impact of the problem.

In totality, this information would determine whether the situation is substantial enough that it justifies mobilizing the resources to produce and implement a program of instruction. Assuming that the problem meets a threshold for an initiative, the next step is to determine the cause of the problem.

II. What is the cause of the problem?

The second phase of Needs Analysis involves investigating the cause of the proposed problem. Rossett (1987) describes four conditions to determine if the problem is a performance issue:

  1. Lack of skill or knowledge: Even if the individuals want to perform the task, they do not have the ability.
  2. The environment is an impediment: Individuals have the knowledge or skills, but other factors prevent them from performing the task (bad equipment, poor communication, organizational chaos, distractions, etc.).
  3. Lack of incentive: The culture of work does not provide a rewarding experience for performing well.
  4. Lack of motivation: Individuals do not recognize the value of the working effectively.

Upon investigation, you may conclude that solving the problem may have nothing to do with training or instruction. Here are some examples related to each cause:

An environmental problem: An employee of a college is expected to enter student application data at a rate of completing 10 in an hour, but they only complete five because of interference from other duties such as answering the phone. Or perhaps the ergonomics of their work environment makes them unable to work efficiently. The employee doing data entry knows how to type and where to enter the data but there are other factors involved that impede performance. Even with a program of instruction, it will not show any performance improvement.

An organizational problem: An employee is assessed in their performance based on evidence of their leadership in initiatives to enhance customer satisfaction. However, even with having submitted numerous proposals for new initiatives, the organizational structure does not provide a pathway for these proposals to be considered by executives for implementation. In this case, the employee is capable to fulfill the expectations of the position, but the initiatives cannot be launched due to an organizational impediment.

An incentive problem: An employee is capable of performing according to standards, but is not compensated accordingly compared to others who perform poorly but are compensated the same.

A motivation problem: An employee does excellent work but resists changing from an inefficient technique that requires cumbersome logistics. While the result of their work meets standards, it could be performed more efficiently if they used a more modern communication system, such as video chat instead of in-person office appointments. In this case, the employee is not motivated to change their method of work because of resistance to change, not due to lack of skill in the work itself.

It is possible, too, that a performance problem may include a combination of factors that involve some instruction and some organizational changes. Your investigation into causal factors informs the range of solutions required to resolve it.

III. What is the “gap”?

The third phase of Needs Analysis involves establishing the basis for the desired status of performance, the actual status of performance, and then describing the discrepancy between the two as a statement of need.

Determining the Desired status: The Desired status of performance may be expressed by stakeholders using concise figures like, “We need to improve output by 10%,” or it may be expressed in a more generalized way like, “We need to improve performance to meet demand.” Stakeholders may already know in advance the nature of the desired status, or it may need to be derived in some way. The ID must investigate the perception of the desired status since it will be used as a basis of setting Learning Goals.

There is a twofold value to setting Learning Goals. The first relates to describing the expected results of the program of instruction. The second relates to the development of an Evaluation Plan where the goals stated at the outset of the project are used as a basis for gathering data and making judgements as to whether the program of instruction was effective in achieving them. Naturally, if you have clearly defined Learning Goals, then evaluating the effect of instruction will be more easily validated and reliable.

Establishing the Desired status involves interviewing stakeholders to determine the following:

    • What are the specific areas of knowledge, skill, or attitude that matter within the scope of the defined problem?
    • Which areas are more important than others?
    • How should the phrasing of the stated Learning Goals reflect the the capability of human activity?

Determining the Actual status: The Actual status of performance describes the performance of individuals according to the context of the problem. The ID would rely on more concrete data or observations to establish this baseline since Actual status reflects reality rather than an aspirational performance goal.

IV. Defining needs to produce a Learning Goal

Dick, Carey and Cary (1978) provide a simple equation to determine the degree of need as expressed in three parts:

Desired status – Actual status = Need

To illustrate this equation, suppose a health care facility wants staff members to administer 20 COVID-19 tests per hour but they are only able to administer 15 tests per hour. There is a gap of 5 tests per hour (a 25% percent gap) between the desired performance and the actual performance rate. A Learning Goal to this effect would expressed the need as:

COVID-19 test administrator staff need to improve the rate of COVID-19 testing per hour by 25%.

A more context-specific statement can be made as follows:

COVID-19 test administrator staff need to improve the rate of COVID-19 testing per hour by 25% by March 31, 2021.

Note that the Learning Goal does not specify how the desired status will be achieved. A Learning Goal is not intended to express anything more than an objective declaration from which other strategies can be negotiated. For example, numerous strategies can be proposed to close the 25% gap, though some of them may be more or less feasible than others.


Learner Analysis

Learner analysis involves gathering relevant attributes of the learners that are relevant to the organization and the overall training goals.

Typically, the ID is interested in the following learner attributes:

  • Prior/pre-requisite knowledge, skills, or sub-skills related to the nature of knowledge or performance.
  • Attitude/motivation (affective) characteristics that are relevant to the perception of the performance, subject matter, and the training itself.
  • Demographic characteristics that are relevant to the context of performance.
  • Cultural or social characteristics that are relevant to the context of communication.
  • Disabilities or language barriers that are relevant to cognitive engagement or participatory needs.
  • Audience characteristics that are relevant to the tonality of communication.
  • Similarities/commonalities among learners that may inform optimal metaphors or examples used in communication.

The most important characteristic, among all the items listed, is the degree of prior/pre-requisite knowledge the learner already knows before attempting the course or training. This includes previous work experience as well as educational levels (Morrison, Ross, Kalman, Kemp, 2011, Dick, Carey, & Carey, 2005).

It is also important to know about the attitudes each learner brings into the course or training. Beyond the basic knowledge and skills to get started, knowing what type of previous experience the learner has had with this subject matter is important in determining their attitude and motivation. Will they participate with an open mind or will they resistant training? Are they motivated to learn or is there little interest in the topic (Morrison, Ross, Kalman, Kemp, 2011; Dick, Carey, & Carey, 2005)?

A basic learner analysis is presented below in Table 2. It pertains to trainees working for a commercial/industrial roofing company. Note that some of the Information Categories in this sample analysis reflect the unique conditions of the roofing trade and the culture of training within it. The ID has discretion to identify unique Learner Characteristics according to emerging needs.

Table 2. Information categories, data sources, and learner characteristics.

Information Categories Information Sources Learner Characteristics
Entry behaviors Interviews: Foremen, human resource manager Learners have adequate physical skills to begin instructions.
Prior fall protection knowledge Interviews: Foremen, human resource manager Limited roofing experience, including lack of fall protection training.
Attitudes toward fall protection Interviews: Target learners, current roofers The current roofers are aware of the need for fall protection even though they complain about how the harnesses infringe on freedom of movement. The targeted learners are unaware of the need for the equipment.
Attitudes toward potential delivery system Interviews: Target learners, current roofers, human resource manager Learners preferred a hands-on demonstration type program.
Motivation for instruction Interviews: Target learners, current roofers, human resource manager Learners are motivated by the need for a job and are willing to do the training for pay.
Educational and ability levels Interviews: Target learners, current roofers, human resource manager Education: Learners typically have a high school diploma or the general equivalent.

Ability: Learners may have roofing experience but are novices in fall protection.

General learning preferences Interviews: Target learners with preference quiz 100% of the new roofers prefer hands-on learning to reading or watching.
Attitudes toward training organization Interviews: Target learners Learners have no prior experiences with roofing company but do have prior educational experiences. Attitudes are mostly ambivalent with a “wait and see” attitude.
General group characteristics Interviews: Target learners, current roofers, human resource manager Interviews: Target learners, current roofers, human resource manager

As you can see in the findings of this Learner Analysis, the results inform the approach you would take in the design of the instruction itself. The results of this analysis also inform the approach to the Context Analysis.


Context Analysis

As described in Table 1, the Context Analysis answers the following questions:

  • Timeliness: When is the program of instruction needed by? How much time do learners have to complete it?
  • Facilities: Where must instruction take place?
  • Transfer: How will learning transfer to authentic practice?

Each program of instruction operates in a context that informs or constrains its design and implementation. For example, an emergency situation that requires immediate training for medical staff will not have the luxury of taking place over several weeks. A training program that requires access to proprietary equipment cannot take place in a separate classroom.  A training program that takes place during a peak season of production would be counterproductive to meeting customer demand.

Timeliness: Your work with stakeholders in planning a program of instruction must include consideration of the time that is available to produce the program of instruction as well as the time that is available for participants to complete it. Stakeholders will rely on your expertise to describe what is possible to produce and implement given the available time frames. Part of your work, as a professional, is to provide reliable and feasible expectations for the work and its outcomes. Your Context Analysis must state the timeframe for both producing the instructional program and the time that is available to conduct it.

Facilities: In some cases, the choices for conducting instruction must be based on their compatibility with the nature of the subject matter and the facilities required for the proposed program. Your investigation into the nature of the subject matter will determine whether the program is constrained to a specific venue, a separate location, a remote (virtual) arrangement, or a combination. For example, some instruction can take place in a classroom, but other parts may require an on-site venue. Your Context Analysis must state the factors that constrain the learning environment.

Transfer: Your analysis of the training or performance problem needs to consider the how learners will transfer or apply their new skills, knowledge, and attitudes into the authentic situations where they are relevant. Consider what type of follow-up support learners will need to successfully transfer learning beyond the classroom. Given the facilities factors, determine whether learners will have an opportunity to practice the skills or demonstrate the knowledge under authentic conditions. Your Context Analysis must state what learners will need in order for them to transfer their new knowledge and skills to the authentic conditions where they will be applied.

Social context

Beyond the mechanical and practical conditions of a training or performance improvement situation, the analysis of an instructional need must also take into consideration the social context within which the problem has emerged as well as the conditions under which instruction will occur. The ID needs to work closely with stakeholders to gather information about the social environment that might guide how participants feel about


Optional reading: Advanced areas of Analysis

The Analysis stage of the ADDIE Model extends more deeply into the details related to the subject matter itself. Often, an ID will collaborate with a Subject Matter Expert (SME) who can explain the unique details related to the tasks, topics, methods, and performances related to the proposed program of instruction.

Subject matter/Content analysis

IDs frequently collaborate with an SME to learn more about the subject matter that will be the focus of the program of instruction. An SME can explain what is important to know about the subject matter or what would be required for a learner to demonstrate proficiency (Jonassen, Tessmer, & Hannum, 1998). It is as important to determine what should not be covered in the program of instruction as much as it is important to determine what will.

Once the scope of a topic has been established, the ID must investigate the nature of the subject matter itself: What are its characteristics, relationships, and foundations? When instruction is presented to learners, the nature of instruction may influence how the subject matter is perceived. A program of instruction about a procedure would be presented to learners, optimally, in a procedural narrative of instruction. Instruction related to concept knowledge should be presented in a way that is optimal for learning about the structure of a conceptual scheme, the criteria for inclusion or exclusion from the scheme, and the relationships within the components of the scheme.  Subject matter analysis informs the organization of instruction itself.

Praxis analysis

Praxis refers to how knowledge, skills, and attitudes are expressed in the professional field or area of interest. The ID does not need to be an expert in the conventions of praxis for the subject matter, but should have an understanding of the nature of it.

For example, the nursing profession relies upon scientific evidence-based practice published in scholarly journals. A program of instruction for nursing professionals would be mindful of the need for instructional media, resources, and assessments to align to the evidence-based standards. For training related to the workplace safety, the ID should be able to collaborate with an SME with an understanding of how OSHA regulations play a part in performing tasks.

Procedural/Task analysis

Procedural/task analysis involves gathering information about how tasks are performed according to sequences, priorities, decisions, choices, and alternatives (Morrison, Ross, Kalman, Kemp, 2011).

The goal of a task analysis is to break down the main topic into a flow of increments so that each node of the task or procedure can be isolated into a discrete unit of attention. Some aspects of a task are overt (observable) while others are covert (cognitive). AN ID inquiring about how a task is performed would ask:

  • What physical or mental actions must a practitioner do to complete the task?
  • What knowledge does the practitioner need to complete the task?
  • What cues inform the practitioner of errors along the way?

The ID, with an SME, will conduct an analysis to develop documentation of an optimal, or benchmark, performance of the task, then create a flowchart from which instruction will be designed to scaffold learners to achieve it.

Information processing analysis

Procedural task analysis is mainly focused on the analysis of observable tasks. However, many tasks involve complicated cognitive components. Information processing analysis is intended to elicit the mental processes required to perform a task or performance (Jonassen et al., 1998). Similar to procedure/task analysis, the outcome of an information processing analysis would be a flowchart of operations and decisions.

There are several models for deconstructing a task for the purpose of instructional analysis. The GOMS Model is composed of the hierarchical flow of goals, operators, methods, and selection as observed and elicited from an expert (Card, Moran & Newell, 1983). This method focuses on adding goals of why the task is important to the learning outcome.

Another is the Applied Cognitive Task Analysis (ACTA) model which asks the SME to identify three to six broad tasks that must be performed to complete the goal. The SME and the designer then determine the knowledge necessary to complete the task and the goal while simultaneously looking for errors novices may make in the process (Morrison, Ross, Kalman, Kemp, 2011).


Below is an example of an analysis that describes the results of research conducted by an ID. The example pertains to a situation at a fictitious family medical facility that is experiencing a problem due to lack of knowledge about maintaining privacy of client health information according to federal guidelines.

HIPAA Training Program: Analysis

The Acme Family Medical Center PLC operates with a staff of 20 frontline professionals who have direct contact with client medical records. In compliance with the Health Insurance Portability and Accountability Act of 1996 (HIPAA), the Company is responsible for maintaining performance standards to safeguard client privacy.

In response to sustaining this performance standard, Dr. Joyce Jones, Executive Director of Acme Family Medical Center PLC, has initiated an inquiry into reports of staff unintentionally mishandling client medical data in ways which may be in violation of HIPAA privacy rules. The following analysis presents the outcome of an investigation into this situation.

Needs Assessment

Problem: The problem has been reported by Dr. Joyce Jones and affirmed in interviews by supervisory management who conduct periodic departmental conferences. Internal assessment of operations has indicated a pattern of low level improper handling of client Protected Health Information (PHI) mostly in instances of note taking during client intake, but also in other situations where PHI can be heard in oral communication over the phone and in sharing information with family members who have not been designated as an authorized recipient of client medical information.

Interviews and surveys of staff workers indicates a high degree of motivation to comply with HIPAA standards, though the evidence suggests that there is confusion about what kind of information constitutes PHI and which circumstances are applicable to HIPAA privacy. Staff also indicate that the staff handbook mentions HIPAA rules, but doesn’t elaborate enough about how they apply to the specific work conditions at Acme Family Medical Center.

The consequences of this problem may have legal ramifications if a major violation occurred where the Company was subject to a fine or an accreditation risk. Our assessment of the problem has determined that a training course is needed for all staff who are responsible for maintaining PHI privacy.

Goal: The desired performance is 100% compliance with HIPAA rules in all instances where staff are involved in tasks related to creating, inputting, reporting, and transferring PHI.

Needs: To achieve the proposed goal, staff will need to demonstrate the ability to perform the following:

  1. Recognize all instances of information that is considered PHI according to HIPAA rules.
  2. Perform their assigned tasks at the Acme Family Medical Center in compliance with PHI privacy requirements according to HIPAA Privacy Rules.
  3. Recognize situations where the appropriate actions are unclear and ask for help before taking further action.

Staff analysis

In interviews, surveys, and group discussions with Acme Family Medical Center staff and executives, we have ascertained the following characteristics of staff members who would be assigned to HIPAA compliance training:

Prior knowledge/experience: Staff experience varies between 15+ years of experience in the medical field to just over a year, post graduation from accredited education. While all staff are familiar with HIPAA rules, none have expressed 100% confidence in their ability to recognize all situations where PHI is a factor in their work, nor which courses of action they should take to be HIPAA compliant.

Social/cultural factors: All staff interviewed are highly motivated to improve their performance and feel strongly that their work should be reflect well on the Company. Staff feel a sense of organizational community, as practitioners, that lends itself to a social desire to help each other to be successful in their work.

Other factors: A few staff members are bilingual which would enable the Company to consider designating employing them in situations where PHI and client encounters are in languages other than English.

Context analysis

The Company seeks to achieve the training goal as soon as possible since the risk of a violation is evident in every business day where staff handle PHI. However, there is no emergency condition that would require a wholesale stoppage of business operations. The timeframe for the proposed training should coordinate with the typical seasonal peaks in healthcare services, staff vacations, and other holidays. According to Company staff, a timeframe around September would be the most practical time to consider. This would enable the training program to be fully developed and refined in time to meet this launch date. Further discussions with staff and management is needed to agree upon a precise date.

Since the subject matter in the proposed training involves a combination of knowledge and application in practice, we propose that the training venue include some use of the Company facilities, perhaps during off-business hours. Some training for basic knowledge can be conducted outside of the facility or online. The use of Company facilities would be critical to transferring learned skills and knowledge to daily operations and would save considerable expense in renting a training facility.

Social context: Our research has indicated that the positive feelings of organizational community lends itself to leveraging it as an integral design of the instruction. The ability to share differential of experience among a diverse group of participants is advantageous to sustaining high level performance in the long term.

Task analysis

A complete task analysis is provided in Appendix A. It describes an action/decision breakdown of the following tasks:

  • Intake of PHI from clients into the EHR system.
  • Interaction with family members regarding client PHI.

References

Card, S. T., Moran, T. P., & Newell, A. (1983).  The psychology of human-computer interaction.  Hillsdale,. NJ:  Erlbaum.

Jonassen, D. H., Tessmer, M., & Hannum, W. H. (1998). Task analysis methods for instructional design. Routledge.

Rossett, A. (1999). Analysis for human performance technology. In H. D. Stolovitch, & E. J. Keeps (Eds.), Handbook of performance technology: Improving individual and organizational performance worldwide (pp. 139-162). San Francisco, CA: Jossey-Bass/Pfeiffer.

Morrison, G. R., Ross, S. M., Kemp, J. E., & Kalman, H. (2012). Designing effective instruction (7th ed.). Hoboken, NJ: J. Wiley & Sons.

License

Icon for the Creative Commons Attribution 4.0 International License

ADDIE Explained by Albert D. Ritzhaupt and Steve Covello is licensed under a Creative Commons Attribution 4.0 International License, except where otherwise noted.

Share This Book