A Bad Learning Experience (and what we can learn from it)
A masterclass of what NOT to do in creating good learning experiences
Was in two minds about whether to share my experiences with the PMP curse (not a mispelling) I'm undertaking. But thought it better out than in.
[NB: Sorry for the dirty nails - I was doing gardening just before this video!]
Its’ worth noting before this minor rant, that I’m no examination and learning newbie, I’ve recently done my Change management foundation and practitioner examinations, my CIPD L&D level 5 diploma and also completed examinations in advanced Mandarin Chinese, as well as my science/media based degree. Not to mention being a learning professional of course!
I've done around 70+ hours or so of study and associated activity on this so far and am reaching the examination point and I'm in two minds about whether to even bother to continue.
For those who aren't familiar, the PMP is a ‘world-recognised’ certification for project management, with a circa 4-hour examination with 180 questions at the end. The course I've undertaken uses the Skillsoft platform via a learning reseller called the Learning People.
Worst. Learning. Experience. Ever.
I just don't know where to start with what has been wrong with the process but here's a high level overview of my experience so far:
1. The PMI:
I smell a scheme, and it has everything to do with money & nothing to do with really improving the quality of the project management profession. An examination that expires after 3 years, after which you are expected to fork out circa £400 for another examination.
This article provides some further insight as to the ludicrousness of their model, citing the following:
Stringent ‘Eligibility’ Requirements
Extensive study materials -
No definitive exam syllabus
Exam length
‘Difficult’ situational questions
Exam format
Unknown scoring system
It’s not stretching it to say that, having gone through a few practice versions, it is the worst designed exam I've ever seen. And as a learning professional, that’s saying something. Just building on the above, here’s some observations:
Eligibility is based on ‘hours spent’ not capability demonstrated. This is not the same thing. If they truly cared about capability, they would be basing their criteria on this, NOT on hours.
The PMI openly say they write them to be 'ambiguous' so as to catch you out. “Just like real life”? I don’t know about you, but deliberately inserting ambiguous language into abstract concepts to trip people up doesn’t sound like real life to me! It might happen accidentally, but that’s a totally different thing. This means the ‘difficulty’ mentioned in the above is not real difficulty in terms of subject matter but meta difficulty of passing the examination. This is of zero value add.
They claim their tests ‘practical application’. Let me say this strongly: You cannot test practical application through a formal examination!
The fact they are highly opaque in the way that they assess and moderate their exams is a red flag in my mind, and reduces accountability to the corpus of professionals who should become familiar with (and co-create!) that body of knowledge. It also reduces the challenges to their examination moderation and results checking.
On the above point, their (expensive) 'PMBOK', costing around £50, which should tie directly to their exam, is apparently not a 'single port of call' for knowledge. Effectively they have no single source of truth, and they don’t tell you directly how and where each source of truth is used.
Of course, if we see all the above through the lens of a money making scheme rather than a true learning imperative, (for example a long 35 hour course that is very expensive, difficult to pass for all the wrong reasons, and requires renewal) then it all starts to make much more sense. Tin foil hat time? Perhaps…but it bears thinking about.
2. The course provider:
As a learning professional I look at the approach taken by the course provider, Skillsoft and I'm appalled.
There’s around 35 hours of self led content + 35 hours of ‘bootcamps’ + What they call ‘reading’, and a test examination.
There is NO structured, personalised support for your learning experience. You’re given ‘access’ and then left to your own devices (physical and metaphorical).
They have an ‘ask a mentor’ button, but my interactions with them so far I’ve found to be focused around being apologists for their poor course design and for the PMI approach in general and simply regurgitating ‘what’s in their knowledge bank’ rather than really dealing with some of my issues.
If this were a well designed experience, a lack of direct personal support may not necessarily an issue but I'm not even sure you could call their content designed, and certainly not well curated; it's more of a spattering of what feels like unrelated, unscaffolded and randomised content, that overfocuses and repeats itself (often through directly copy-pasted sections of course materials) in some areas and utterly abandons in others.
I reach the examination point and quickly realise that the time I've spent so far on their content probably only covers maybe 30-40% of the end point examination requirements. I query with the provider and they admit that their course "doesn't cover all the content". They point me to a bunch of PDFs they've slapped into their LMS that I "Should just go read". This amounts to thousands of pages of unstructured content….where is the value I should be provided?
Notably the examination is based in three domains ‘people’ ‘process’ and ‘environment’. The course basically avoids most of the process related content, which constitutes 50% of the end grade. How on earth a process-driven qualification can have so little process in it is beyond me. I am great at process, but I’ve yet to see how they have effectively mapped out the end-to-end process and signposted how and what knowledge needs to be called out along the way.
Their 35 HOURS of 'live bootcamps' were rambling webinars of one instructor waffling through a few PDFs and replaying self-led content from their library. They should be ashamed.
3. The course vendor: I wasn’t aware at purchase, but the vendor was not the source provider for the course content. They gave me the impression they provided this service. A total lie of omission. Instead they are simply a reseller. As such they have demonstrated an utter lack of accountability/control for any of the course components, complaints and issues are met with attempts to 'downplay' and gaslight. One comment from their ‘student support team’ was: "We have a high pass rate, so it must be you who just hasn't done enough study". I actually came to them for a change management course (which I also purchased and have passed with a different provider). Looking back the course was actually totally mis-sold to me, using sometimes outright lies to get me to purchase.
Can honestly say - If you're thinking of doing the PMP via a course, at least this one. Don't. If you must do it (and it's of dubious value), then just buy a book and a test prep document. You'll be better off.
That said, to flip this experience. What can we learn about learning experiences from this bad learning experience ?
Learning experiences are holistic. They need to encompass a users end-to-end need, and recognise the systemic concerns that all users of the learning system have. In this case the system is so convuluted that it needs taking apart.
Communication, context and curation, and constructing meaning all count towards . We could call this the ‘4Cs’ of learning experience design.More on these ‘4Cs’ later.
Learning experiences, once they go sour, can go sour very quickly, and like customer experiences are difficult to recover from. But like customer experiences they need that rescue mechanism, we may wish therefore to approach learning experience design like other fields like service design that put a user at the centre.
Old mechanisms of information delivery do not work. Plain and simple. No matter how much lipstick is put on the pig (in this case there wasn’t even that much lipstick).
ADDITIONAL POINTS TO EXPLORE?
More widely beyond learning experiences:
That the learning ecosystem that supports this kind of product is broken and needs revising. One could argue that the certification systems that acredit learners is a system of academic inflation based on business outcomes, not learning or performance outcomes.
That we as B2C purchasers of courses need to be more savvy (even I was taken in and this is my profession!)