Your secret weapon to assess domain knowledge is: Domain Quiz!

9 min readMay 17, 2019

On my talk at KanDDDinsky conference in 2018 I introduced something called Domain Learning Framework and talked about its elements as well.

One of the element is the Domain Quiz. I’ve tried to explain there what it is but obviously during a talk you cannot spend that much time on the practical things in too much details. Since then I had the pleasure that I got mentioned with this practice by Nick Tune more times in various talks of his.

Recently I got questions whether this Domain Quiz thing is documented anywhere at all? And…it is unfortunately not (except my previous domain quizzes at various companies) yet, but that’s why I’m writing this up for you now. Because I wanted you to see that you shouldn’t be thinking of any difficult assessments here, or hard to fill questionnaires…

What is Domain Quiz?

A relative short assessment of people’s current domain knowledge followed by a common evaluation session.

And, if you ask me, it is also another fun way of learning about the domain. Or at least you can make it fun for the people and believe me they will bug you to do the quiz more often. 😊

You must remember the exams back in time how educative it could be when you had the chance to look at your incorrect answers and learn about what you’ve mistaken. This way you really can learn way much more in my opinion than just listening to some product walkthroughs or listening to the domain expert speaking.

The Purpose & the Reason

The goal is always the same here: you want to learn about your team’s (or colleagues, etc.) current domain awareness in order for you to know where to improve it.

You can do the quiz either for the whole domain (even maybe for organisation-wide domain) or just for a smaller, specific part. It needs to be you who knows what is needed and why.

However the reason why you want to learn more about the level of people’s domain knowledge may vary. Some of the clues can be like:

you simply get too many questions from the team on how things work
many times the features don’t even pass the dev test phase
you realize that QA spends too much time with testing even if there is some simple improvement made in the system
when you are not available people ask the team about the system behavior and they have no idea about that
there is a specific feature you need to design and implement with the team, but first you want to have an understanding on the team’s awareness about that area in the system
…

The Audience

Your audience really depends on the purpose. You can do a quiz with your team, with individuals who are for instance stakeholders in a project and working towards the same goal, you can do it with the management even. 😯

Or do cross bounded context quiz with multiple teams and you can make it a competition at your company. 👑

You can decide whether or not you do anonym quiz. I would recommend the anonym one, as according to my experiences people are more relaxed if they know that their names won’t be there attached to their results. And why would you need their names? You don’t, really, because you want to identify the areas (not individuals) where the team as a whole needs the improvement. However sometimes you might want to identify who reached the maximum points in quiz, you can read about this below.

The Timing

You can do the quiz anytime! I don’t think it is something that needs to be done around milestones or in specific sprints or quarters. Anytime, and which is maybe more important: regularly (read below).

In terms of the lengths I would aim not longer quiz than 20 mins (which probably means 10–15 multiple-choice questions max).

Content

I think this is the hardest part because this is really dependent on the context. I have bad news: I obviously cannot tell you exactly what questions to ask from your team or from the wider audience about your domain and also I cannot show you exact examples from my domain quizzes (because those are strictly work related). But what I can do is to provide you some guidelines for putting the questions together:

👥 End Users

Definitely ask about the end users of your system. And remember to provide seems-to-be-valid but incorrect answers so when people will see your feedback they will be really able to see the differences why e.g. the sales associate and not the sales manager is the end user of your product.
And don’t ask like “Who is our end user?” but ask like

“Who is the person we provide the most value for?”

Here you can be tricky with the answer options because of the word “most”. Maybe you provide value for more personas but there is one persona who has the most value out of the product..

💰Value

Ask about the value, ask why users are using your product or why they are part of the certain domain flow. Here you can be really creative with the answer options. Be careful and ask without referring directly to the end user because then your first question worth nothing :)

📦 Different features / functionalities / behavior

If there are specific feature sets or functionality groups you want to find out your team’s domain awareness about, go ahead and ask about those specifically.

If there are reports or analytics or some special calculations in the domain eco-system you want to check your team’s knowledge about, it is good to include that too. But in a way like you write a short story including the input numbers into the calculation and as answer options you can provide possible numbers as the solution. Here you can obviously operate with only one correct answer (unlike the other questions where you might want to have multiple correct answers or “All of the above” kind of thing).

Evaluation & Findings

You should analyze group results and look at the weakest areas where the team really needs improvement. You should do common session/s with the team after you analyzed the results to address these areas together.

It doesn’t matter whether you do the quiz online or on paper, the evaluation together with the team needs to happen. That’s the main point in it, that’s where everyone can learn the most.

If you found out that they barely know the domain don’t worry! There are plenty of things you can do.

Here are a few I would definitely recommend (these are all parts of the Domain Learning Framework):

Domain Flow Simulation (mob session) — you will be able to learn about this in a future blogpost
Dog Food Context (regular usage of system/s) — you will be able to learn about this in a future blogpost
Pairing up with the domain expert (pair programming) — you will be able to learn about this in a future blogpost
Event Storming Workshop (workshop) — learn more about EventStorming or read my blogpost about EventStorming

I promise you can learn about these later in a future blogpost in more details. 😌

Tips&Hints

🎯Emphasize Goal

You only want to assess in which areas the team needs to improve their domain knowledge. Always emphasize that this is not about individual results and punishment. When people hear words like “quiz” many of them think it is some assessment to judge them and there will be consequences if they don’t do 100%.

🏆Add Prize

I suggested above to do anonym quiz. However there is one case when you (and team) might want to identify the one who did the best in the quiz: when there is a prize 😊You might want to make sure you show the prize in advance to the team as a teaser. Prize can be anything, a small thing like a piece of chocolate or even some bigger things like half a day off or anything like that, it depends how much you want to motivate people and how important it is for you to learn about the domain awareness.

In this case — when you have a prize and the quiz is not anonym — you need to associate points for each answers (how much it worth). I also would recommend to do the quiz on paper here: print the quiz (one for each person), book a meeting room (in advance), hand out the papers, set the time (I think it is good to have a workshop timer for this purpose) and when the time is up and team is ready, start to talk through the questions and answers immediately. Everyone has their own filled in quiz in front of them, you might want to prepare slides maybe for taking them through the answers or it can be just a discussion without any slides. It doesn’t matter, the main thing here is that they need to be able to evaluate their own quiz with your guidance (and you should be able to remember the scoring 😊).

When finished with evaluation, everyone can count their points, and you can start asking around: who has the maximum score? And give the prize to the winner. 👏

📚Spend Time on Preparation

Send out the domain areas in advance so giving the opportunity for people to learn. If you will have a prize, make sure you mention this beforehand, they will be more eager to learn and prepare. Even with this the team’s domain knowledge can be improved already.

Spend time on figuring out “good incorrect answers”. Incorrect answers can be very educative if they are phrased properly. You can start by thinking of how you will explain why it is not the correct answer. Sometimes we can learn more about things if we see what things are NOT.

Also make sure your answers are not predictable. Then the quiz would probably be just a waste of time for everyone, and you don’t want that.

🔧Tools

I’m sure there are planty of tools you can put a quiz together with. I usually use Google Forms as I’m very satisfied with its behavior. It is very easy and quick to put the quiz together (after you’ve already done the hardest part: figuring out the questions), I really like that I can do feedback (whenever someone submits the quiz they immediately see the feedback about correct and incorrect answers and the reasoning behind that I provide).

No matter what tool you use, always make sure…

that you provide feedback at the time of the submission
you provide reasoning both for correct and incorrect answers
the answers are not predictable
you test it thoroughly in advance — send yourself the link and fill in the quiz and check the feedback etc. before you send out to your people. Step in the team’s shoes and try to think how will they feel when they see that particular feedback or question.

If you want to have a quick look on a dummy quiz where you can have an insight into Google Forms’ capabilities in terms of a quiz just check it out and play around with it. Now I enabled you — as the one who fills in the quiz — to have a view on the other results and stats as well as your total score etc. Normally I don’t allow people access to these the common evaluation session. So I like to check and assess the domain overall knowledge first, and only share the cummulative results with the team in a meeting where we sit down together and discuss it through.

🔄Do Follow Up

And what you shouldn’t forget is that you need to quite regularly do a domain quiz to see the awareness with the updates and changes in the domain. This means more work for you because it is not about having the same domain quiz all the time but you always need to come up with new questions. But obviously 2–3–4 questions are totally enough as a follow-up if you’ve already made sure earlier that the basic domain knowledge is there and now you just want to assess the awareness with the newer things.