The following articles, written by Sean Triner, Co-founder and Director of Pareto Fundraising and Pareto Phone were originally published in The Agitator column of Fundraising and Philanthropy Magazine’s e-newsletter.
Earlier in the year I was fortunate to be invited to speak at the Institute of Fundraising (IoF) Convention in the UK, and I recently received the audience ratings for the presentation. As an egotistical, needy, must-be-loved, ‘please like me’ person, audience feedback is really important to me (especially when it’s positive, as the IoF evaluation was). But the feedback format got me thinking, and really, the whole scoring system is seriously flawed.
Conference organisers need an objective way of measuring their speakers’ performances. They really need to be able to get the best speakers back and establish a system of good presenters to build their reputation. Seems obvious. The problem comes from a conflict between the only real purpose of a fundraising conference — empowering individuals to make more money for their organisations — and the fact that people want to enjoy themselves, be entertained, laugh and learn. If you ask, people will always say learning is the priority. But this is not reflected in the scores.
Basically, most people score emotionally. They will score a speaker higher if they like them, they will score higher if they have fun. But worst of all they will score them lower if they disagree with the speaker. The last point presents a huge challenge for any speaker trying to challenge the prevailing paradigm, which of course we need to do if we are to create change.
Now bear in mind a few things:
- Most fundraising conference speakers are not paid and are not professional speakers
- A large proportion of conference attendees are usually first-timers and people new to fundraising
- Scores are usually generous — speakers rarely get the worst possible ranking
- Session attendance can vary from a dozen to hundreds, so a couple of out-lying scores can have a dramatic impact for some people
As I mentioned before, I have just got my scores from the IoF. The marks for each session were broken into two parts, one for content and one for presentation. Content and presentation scores were always within 1% of each other, and yet the marks between different sessions varied much more. So in other words, most people weren’t really distinguishing between content and presentation.
The best learning comes from highly skilled teachers or trainers who really know their stuff, give direct practical information, are willing to adequately prepare before the conference and are engaging and fun. The problem is they are very rare — hence the international conference circuit has the same old names cropping up.
So what can we do about it?
Well, we need a paradigm shift about how we evaluate conference speakers and the decision-making process about how we invite them back.
The organisers of the key fundraising conferences should come together to develop effective and standardised conference evaluation policies and procedures. This includes organisations such as the Institute of Fundraising (UK), Resource Alliance, Association of Fundraising Professionals, Fundraising Institute of Australia and Fundraising and Philanthropy Australasia Magazine. Leave egos behind and do what’s right for the fundraising community. They should develop:
- A much more thorough and data-driven approach to the conference and speaker evaluations. The attendees are customers and should be analysed like a charity would analyse its donors.
- A way to gather more data on attendees — before the conference — to find out about their level of experience, area of work etc, and then analyse it to see who goes to which sessions.
- A process whereby conference organisers can log who came to what session to put post conference feedback in context.
- An evaluation system that has more emphasis on learning outcomes. This means scoring sessions on more than just content and presentation.
- A process that measures scores in grades of ten rather than four. Marks out of ten (rather than ‘poor, ok, good, excellent’) create more significant gaps on measures and also make it easier to provide feedback to speakers and conference organisers.
- Evaluation forms that collect more useful written feedback that can be passed on to speakers.
- A process that takes into account attendance numbers when looking at average scores.
- A process for following up all attendees using something like Survey Monkey to ask people what they have done with the learning.
Once these processes are in place, we then need a better way to publish and share scores, information and analysis amongst speakers and conference organisers. The challenge is to get speakers to disavow anonymity and share their results with each other, conference organisers and even with conference participants.
These are just some ideas — but I know many of you have lots of experience as a conference attendee or speaker, so I would very much welcome your views and ideas.
Got something to say about this article? Tell us by clicking here
About the Author – Sean Triner
Sean Triner likes to cause controversy “to get people thinking and planning properly, and to stop doing things the way they always did just because they always did!”
Sean co-founded and continues to lead the international Pareto Group, one of Australia’s most dynamic fundraising and marketing agencies with offices in Australia, New Zealand, Canada and Hong Kong.
Sean has been a presenter at some of the world’s best-known fundraising conferences including the IFC in Amsterdam, FIA, and the International Workshop on Resource Mobilisation (IWRM) in Malaysia.
