By Dave McCool
With National Career Development Month landing in November, it seems like an appropriate time to share the results of a recent pilot test of courses created to help people all over the world advance in their careers. SkillBuild by Muzzy Lane is a series of online short courses we developed in partnership with Education Design Lab. The courses are built on the Lab’s foundational 21st Century Skills framework, and their purpose is to support higher education institutions, employers, and learners in developing, assessing, and verifying soft skills such as critical thinking, collaboration, oral communication, and resilience.
The SkillBuild suite enables learners to practice, master, and earn credentials for soft skills, and employers can integrate these courses into their learning and development programs to support employees in upskilling or reskilling. Higher education institutions are also using the courses as part of their transition to more skills-based pedagogy. Here’s a quick summary of our pilot, along with some key takeaways that can be helpful to anyone working in the skills community.
The pilot focused on the course teaching critical thinking, specifically looking at a sub-competency called “Gather & Assess Information.” In the active learning course we piloted, a series of role-play simulations assessed learners’ abilities and provided opportunities for practice and remediation. A virtual coach guided learners through the course and adaptively delivered appropriate challenges. Along the way, learners addressed real-world scenarios and made choices that built on each other. As they saw the effects of their decisions, they learned to course-correct. When learners successfully completed the course, SkillBuild awarded them a “Gather & Assess Information” microcredential that they can showcase on their resumes.
The pilot included 99 people representing many demographics, including individuals interested in earning a badge for career advancement, college professors, and company HR representatives. The goal was to gain a better understanding of what learners experienced while using “Gather & Assess Information.” To do this, we conducted a series of 1-on-1 interviews, and also held discussions with larger groups. We asked pilot testers the following questions:
Were you able to complete the course?
Did you encounter any roadblocks?
Did you find the course easy to use?
Were the objectives clear?
What did you like most/least about the course?
Did you find the course easy to navigate?
Was it sufficiently rigorous?
Did you find anything confusing?
Was the length of the course what you expected? Longer? Shorter?
Was the coach helpful?
We received positive feedback that pointed out the value of assessing skills using simulations:
Pedagogy: “underlying pedagogy seems sound.”
Role-playing: “The roleplay is an engaging and effective way to learn these skills.”
Relevance: “The information in this product...will help me in my job.” “I also appreciate the real-life nature of the product.
Credentialing: “I really like that you earn a credential.”
Confidence-building: “I feel more confident now about using these skills.”
Pilot testers, one of whom stated flatly, “I didn’t finish the course,” also identified several areas for improvement that developers and instructional designers would do well to keep in mind:
Getting stuck: “When I got stuck, I wished the coach would give me an extra nudge in the right direction.” “It was a little discouraging to be sent back to the same exercise until I got it right.”
UI issues: “I had trouble finding the quantitative data I needed to make a recommendation.” “Some screens have a lot of text on them.”
Additional data gathering: “It would be good if the product could detect hesitancy in users.”
In response to this constructive feedback, our team is working on a number of improvements to SkillBuild including:
Clarifying sign-posting so learners always know where they are in the course;
Reducing the amount of text on screen (especially important for learners using the product on smartphones and tablets);
Providing additional nudges and encouragement when learners get stuck;
Clarifying debriefings by the virtual coach;
Streamlining transitions into new challenges; and
Enhancing rewards for completing challenges.
The goal of all these updates is to help more learners finish the courses and earn the credentials. As we continue to iterate, we may conduct additional pilots or gather information in other ways, but we are committed to building these courses in collaboration with the people who use them to advance their careers. I hope these results spark some ideas for you, and Happy National Career Development Month! David McCool is President and CEO of Muzzy Lane, a company that was recently awarded 1EdTechTM 's 2022 Gold Learning Impact Award. Since founding the company, Dave’s goal has been to build technology that empowers authors to create compelling online experiences and helps students practice skills with guidance and feedback. Dave was previously involved in the founding of 2 successful startups. He graduated from MIT with a BSEE in 1987. He can be reached at dave@muzzylane.com or LinkedIn.