Home | about | articles | assessing-your-readiness-implementation-learning-analytics-making-start

Assessing Your Readiness for Implementation of Learning Analytics: Making a Start

Share it now!

Lindsay Pineda, Senior Implementation Consultant, Unicon, Inc.
Patrick Lynch, Teaching Enhancement Advisor, University of Hull, UK

As an institution, you may find yourself asking, “How do I know if we are ready for learning analytics? Is there a way to ‘feel out’ where we are before having someone come onsite for a more official assessment? What kinds of things can I do on campus to prepare for an on-site Readiness Assessment?”

One of the main advantages of engaging in an on-site Readiness Assessment is that the event is tailored for your institution’s particular requirements. Our clients have confirmed for us that the “gold standard” in this work is to bring in people from outside of the organization to help facilitate this process. Institutions have told us:

  • “This is honestly the first time we’ve had all of these people together in one room to talk about a common goal.”
  • “Bringing someone in from the ‘outside’ onsite allows for an impartial view of the institution and it’s not muddied by an emotional connection to one thing or another. That is extremely valuable.” 
  • “I have no idea how I would have been able to convince this many people to be in the same room together for this long without using the pull of Consultants coming in to assess us.” 
  • “With someone coming in, no one seems particularly ‘suspicious’ about the intent behind the workshops. People are curious about who is here and what they intend to help us do so they actually show up.”

Making a Start

Through our experiences, we did find that there are activities an institution can undertake as “prep work” for the facilitated on-site Readiness Assessment. Getting people together for a sustained period is difficult. Getting disparate groups together is challenging and organizing senior leadership into any narrow time frame can be near impossible. We experience these obstacles ourselves while planning on-site visits with institutions. 

In this article we present two activities that will help an institution better prepare for a shorter, more focused, external engagement with their on-site facilitators. This also provides the facilitators with richer information to help deliver the most value while onsite at your institution. 

Areas to Investigate

One of the key elements of readiness is to consider the fit of a learning analytics project within the culture of the organization. This includes policies, processes, and practices that may be affected. Taking a broader view can help identify where existing projects and processes can support the initiative. For example, we have encountered change fatigue in a number of institutions. Wrapping activities together into one broader program is a great way to reduce the perceived number of changes, as well as help realize the advantages of interrelated projects. For example, an existing project at focused on ethics and privacy policies could also include those policies needed to support a learning analytics initiative.

Stakeholder Involvement

Given the broader approach to implementing learning analytics, a wide stakeholder group needs to be involved. Assessing readiness acts as a solid start to the project. It is a way to introduce colleagues to the goals of the project and benefits of implementing the initiative effectively. The assessment also assists with gaining buy in from the start. Key staff groups we have identified are:

  • Learning and Teaching support/development
  • eLearning/TEL
  • IT Services
  • Academic Tutoring/Advisement Services
  • Student Support Services
  • Research and Planning
  • Registrar’s Office
  • Key Thought Influencers 
  • Senior Leadership (“Decision Makers”)
  • Faculty/Teachers
  • Library/Information Services

And last, but by no means to be considered least:

  • Key Student Union/Government Representatives

For each area, as appropriate, we would suggest having both academic staff as well as administrative individuals involved.

Having this broad range of departments together offers a good overall representation of the institution and allows for sharing of impact and ideas in a broader manner. Each of these areas brings a different perspective. It includes the individuals who will see the effects on the institution's bottom line, who will carry out the actual work, who will provide interventions, along with the end users. 

Activity Recommendations

The two activities we recommend are: 

  • Setting Institutional Goals and Objectives
  • Determining Challenges and Obstacles 

The first activity can be used as a starting point for goal setting, which will be iterated upon in subsequent activities. These two activities are specifically aimed at narrowing down the institution’s short and long term goals while identifying any major challenges/obstacles. This allows for the on-site efforts to center around the already identified goals, challenges, and obstacles.

Activity #1: Institutional Goals and Objectives

  • Why Is This Important?

    Identifying short and long term goals upfront, with several people involved, clarifies the institution's purpose for engaging in a learning analytics initiative. Setting short-term goals provides more attainable, smaller wins to be celebrated along the way. Long-term goals provide guidance and direction for an institution's strategic planning, budget allocations, and resource assignments. These discussions need to remain focused on realistic goals that can be accomplished within the agreed upon time scales, resource availability and senior leadership requirements. There will likely be goals that are proposed that may not meet the above considerations. This is acceptable for discussion, but may not be acceptable for the final outcome. To ensure the discussions remain focused, some questions to ask might be:

    • • What are your department’s existing strategic goals? How will a learning analytics fit in with those existing goals/requirements?
    • • What are the institution’s existing strategic goals? How will the learning analytics goals fit into those existing goals/requirements?
    • • Are the identified goals in line with senior leadership’s strategic vision? If not, how can we incorporate them?
  • Intended Outcomes of the Session

    This activity is intended to bring together a shared understanding of what learning analytics means for your institution. Different departments and individuals have varying definitions of what a successful outcome of a learning analytics initiative looks like. It is important to establish a collective understanding and vision at the start to ensure communication is consistent throughout the initiative.
  • Duration

    This activity takes some time; two to three hours is a reasonable time to allocate. This allows ample time to discuss the goals and reach a mutual agreement. Keeping the discussion focused on what can be done is important. Ensuring the conversations are positive sets the tone for the overall project.
  • Setting and Structure

    This activity is best carried out in a larger group setting. Separating the larger group into smaller breakout groups of four to five people each is particularly important to mix up the departments. This ensures the groups are not homogenous and each person gets the opportunity to hear different viewpoints. This ensures certain individuals are hearing each other and understand the overall impacts to all areas, not just their own. After allowing the small group discussion, facilitating an active conversation in the larger group provides a check for collective understanding of the institution’s goals.
  • Resources and Supplies

    For this activity, the resources and supplies are minimal. Flip charts are a good way for the smaller groups to write down their goals so they can be shared with the larger group. Pens and sticky notes are also useful.
  • Prerequisites

    • • If senior leadership representatives are unable to attend the collective activity, it is important for a Project Manager or Lead for the initiative to meet with each of the impacted stakeholders prior to the scheduled activity.
       Gaining insight into the high-level strategic goals and their commitment for the initiative overall is important. These insights can be shared with the larger group during the scheduled activity.
    • • Commitment from the invited stakeholders should to be obtained prior to conducting the activity. If you do not have a strong commitment for attendance and participation, the activity will not be successful.
  • Who Should Be Involved?

    A broad range of stakeholders should be invited. One to two individuals from each of the affected departments and areas needs to attend. Some of the roles we have identified that would be helpful are:

    • • Heads of Department
    • • Executive Deans
    • • Deans
    • • Directors
    • • Managers
    • • Academic Tutors/Advisors
    • • IT Individuals
    • • Faculty/Teachers/Professors
    • • Project Management Representatives
    • • Student Support Services Staff
    • • Research and Planning Staff
    • • Registrar’s Office Staff
    • • Senior Leaders (President, Chancellor, Vice President, Vice Chancellor)
    • • Key Student Union/Government Representatives

    These are just suggestions so it will be important to assess the stakeholders that will be affected by the initiative.

  • Outputs

    It is not necessary at this early stage to have a polished product, as the goals will be revisited in future activities. A collective vision and understanding of the overall impact to each department will begin to be identified.
  • Summary of Hints and Tips

    As we have conducted these types of sessions with institutions, we have learned a few tips along the way to help make things a bit easier:

    • • Have two people facilitate the activity. This allows for one person to take notes and write down non-verbal observations and nuanced conversations
    • • Break the large group into smaller groups. Remaining in a large group is impractical for this activity
    • • Join in the discussions. Listen to what is being said within the groups and begin to find similarities among them. Encourage groups to explore issues more broadly and ask questions of each other.

Activity #2: Challenges and Obstacles

  • Why Is This Important?

    It is important to discuss some of the challenges and obstacles that will need to be addressed to achieve the identified goals. Implementing learning analytics will take combined effort and significant commitment from many people. Encouraging honesty upfront about the challenges and obstacles for each department generates transparency and a shared sense of purpose.

    • • Have two people facilitate the activity. This allows for one person to take notes and write down non-verbal observations and nuanced conversations
    • • Break the large group into smaller groups. Remaining in a large group is impractical for this activity
    • • Join in the discussions. Listen to what is being said within the groups and begin to find similarities among them. Encourage groups to explore issues more broadly and ask questions of each other.
  • Intended Outcomes of the Session

    This activity is intended to bring together a shared understanding of what challenges and obstacles are present and how they relate to the goals. Each department and individual will have different concerns about the initiative, including how they will be affected.
  • Duration

    This activity takes a minimum of four to five hours. This allows the groups to first identify challenges and obstacles for their department, group them into common themes, and discuss potential mitigation strategies for each of the identified themes. It is paramount to remind everyone that his or her input is important, valued, and that not every challenge or obstacle identified has an easy solution. This activity is meant to uncover issues and brainstorm, as a collective group, some potential ways of addressing them.
  • Setting and Structure

    We would recommend carrying out this activity in a larger group setting with the same representatives that were in attendance for Activity #1. It is very important to divide the larger group into smaller break out groups once again to allow for extensive discussion time. Continuing to mix up the individuals from the departments is also very valuable in this activity. Once the smaller groups’ challenges and obstacles are shared with the collective group, the facilitator can begin organizing them into themes.

    Some of the main themes we have found are:

    • • Data issues/concerns
    • • Student issues/concerns
    • • Staff issues/concerns
    • • Ethics and privacy issues/concerns
    • • Resource availability, allocation, and skill level issues/concerns
    • • Buy in issues/concerns

    This is certainly not a comprehensive list and each institution will have its own themes that begin to surface. What is most important is that the whole group agrees on the themes identified so the focus is on collaboratively brainstorming potential solutions to each issue.

    The mitigation strategies will differ between institutions. It is important to keep the solutions to a more realistic tone; however, thinking outside the box is highly encouraged. Flip chart paper can be used to provide a place for recording the themes, challenges, and mitigation/solution discussions. At this early stage we also found that conversations often explore ownership of the thematic and mitigation areas.

  • Resources and Supplies

    Flip charts, sticky notes and pens are the best supplies to have on hand. Sticky notes could be used to write down identified challenges and obstacles.
  • Prerequisites

    Again, if senior leadership representatives are unable to attend, their challenges and obstacles should still be collected.

     

  • Who Should Be Involved?

    It is important to ensure you are inviting the individuals who were in attendance for Activity #1 to this session. The consistency of commitment and involvement is vital for continued buy in and a successful initiative.

     

  • Outputs

    The main outputs of this activity are that of a shared development and understanding of the institution's main themes as they relate to the challenges/obstacles of implementation. Collective agreement of the themes and potential mitigation strategies create advocates across the institution along with an overall awareness of a change happening within the university or college.
  • Summary of Hints and Tips

    Some additional tips for this activity:

    • • Again, having two people conduct the activity is extremely valuable.
    • • Break up the larger group into smaller ones, as it is easier to manage.
    • • Allow the discussions to happen freely, only providing guidance when needed. After successfully facilitating these types of activities at several institutions, we found that stifling the conversation or trying to “rein it in” could be more harmful than helpful. We saw the collective exercise of thinking through a challenge or solution drew the groups closer together and created a cohesive “super team.” This is very important to establish, as it will create advocates for the initiative moving forward.
    • • Sharing out the findings after the session is crucial. Taking pictures can also be of value to those who are more visual.

Why Use an Outside Consultant?

While conducting the above activities can be very valuable for an institution to glean some initial insights, it is not realistic for an institution to undertake an entire Readiness Assessment on their own. The outcomes from the “prep work” activities above will provide the outside consultants with baseline information to build upon with institutional staff and students while onsite. Our recommended “prep work” activities above are meant to assist an institution for the preparation of an on-site, facilitated Readiness Assessment; not to replace it. There are several additional activities experienced facilitators will guide an institution through. Institutions have advised us that having someone come from the “outside” creates a sense of urgency and importance around the learning analytics project and the Readiness Assessment activities. This can be valuable to leverage.

Some of the other valuable areas that we provide for institutions as experienced Readiness Assessment facilitators are:

  • Introductions to learning analytics, including real world experience and examples
  • Student requirements scoping 
  • Policies, processes, and practices insights 
  • Ethics and privacy considerations 
  • Technical/data considerations 
  • Intervention considerations, guidance, and management

These areas, and many others, are discussed in a guided, collaborative, and facilitated manner through workshops, sessions, and activities. The findings from each Readiness Assessment are shared in a confidential final report format that is meant to provide broad insights into the institution's collective readiness. There are also recommendations made for next steps in the learning analytics initiative journey.

We heard from several institutions that an on-site facilitated Readiness Assessment is an essential first step to engaging in a learning analytics initiative. One institution told us, “An on-site facilitated Readiness Assessment is an efficient, effective, and valuable part of starting a learning analytics initiative.”

Resources:

We have provided some resources below regarding additional insights and services as it relates to readiness. 

Unicon
Unicon offers a Strategic Planning / Readiness Assessment consulting service that was born out of the engagement with Jisc over the past two years. Unicon has taken several aspects of the offering done in the UK and made some additions to offer it to U.S. higher education institutions as well. The primary facilitators for the Unicon offering have extensive experience in providing the on-site Readiness Assessments at varying institutions of size, type, and location. 

For more information or to engage with Unicon for a Readiness Assessment, contact a Unicon Solutions Consultant.

Learn more about Unicon’s services for learning analytics, and access additional articles written by both Lindsay and Patrick. 

Lindsay and Patrick also participated in a video series regarding institutional readiness. 
 

Jisc
Jisc has provided an on-boarding guide and checklist for institutions who wish to engage in their own discovery on-boarding process. These guidelines are aimed at institutions who are interested in implementing the Jisc learning analytics service and are looking to undertake some initial steps to prepare for implementation of the full service, the student app, or to gather data in the learning record warehouse.

Discover more about Jisc's on-boarding guide and checklist.

Lindsay and Patrick are also part of the Jisc Readiness Assessment offerings to institutions and have articles published on the Jisc Effective Learning Analytics blog site as well.
 

EDUCAUSE
The EDUCAUSE benchmarking service helps CIOs and other campus leaders to measure progress on campus wide strategic initiatives. The service provides capability reports comprised of maturity and deployment indexes for analytics, culture of innovation, e-learning, IT governance, IT risk management, information security, research computing, and student success technologies. They have published some information on the analysis on their various maturity indices, which is designed to test for “readiness” for analytics in general (not necessarily institution-specific). This information is available on their website.

Useful Reading:

 

 
Yes