There are many different ways of organising the adjudicators’ briefing day, and indeed it’s quite healthy if the format changes from year to year. However, it has become an increasingly important part of the Championships as we attempt to standardise judging perspectives and improve the competition.
Timing and structure[]
1999, 2001, 2002 and 2003 saw a full day set aside for training, on the basis that there is a huge amount of material to cover and that reminders are useful. On the other hand, many ‘experienced’ judges felt that much of it was re-treading old ground and could be dispensed with – so 2004 saw a return to a shorter session, namely a half day. In 2005, the training session was for two hours after dinner and after a full day at Banff. This proved to be far too short so a return to a half day took place in Cardiff in 2006. The general consensus was that it was still a little short, especially for new adjudicators. In 2007, there was a return to a full day of training which included, as in 2002 and 2003, a test whereby the adjudicators all marked a DVD debate and handed in their marksheets which were then reviewed by the CAP. There’s no right answer, but there are certainly key areas to which everyone would benefit from refresher courses or introductions. These include: use of the marking scheme, filling out the ballot, evaluating fair definitions, marks given to POIs, penalties for time or other infringements, giving adjudication speeches, giving feedback to teams, behaviour at schools, workshops, complaints procedures, feedback forms to be completed by team coaches and an explanation of the judge allocation system.
First-time judges would benefit from more intensive training on the differences between Style, Content, and Strategy; the need to abandon pre-conceptions of debating style and welcome the differences between styles rather than penalise them; and a general question-and-answer session about the Rules, Notes and organisation of the competition. This might be done in a separate room while more experienced judges talk about giving workshops and adjudication speeches.
In addition, all judges must be told to re-confirm the nations they are conflicted with. This has become a growing issue, as an increasing number of judges watch unofficial spar debates between teams prior to the commencement of the tournament.
However the CA plans the day, remember to include coffee and lunch breaks at reasonable intervals.
See also[]
- Discussion page for Trevor's views on other crucial areas.
Inclusion of coaches[]
Some training days have been restricted to adjudicators, others have required coaches to attend, and for some, coaches have had the option. There are certainly areas which would benefit from a common understanding between coaches and judges, such as definitions, expectations of POIs and so on. It would also be useful to have an opportunity to address coaches about how they interact with judges – not berating them in front of teams, listening to feedback with respect, completing feedback forms, what to do if they have a complaint and so on.
Preparations[]
Once you’ve made the decision on how many people to include, and whether to split them into groups, make sure you have enough rooms with enough chairs organised in advance! Judges should also have paper and pens, whether they bring them or you supply them.
Session presenters[]
It used to be the case that about 5 people had been to enough WSDC competitions to be able to deliver training sessions. Now that number is probably 20 or 30. It’s a good idea to split topics up and ask different people to present them, for the sake of audience concentration and also to make experienced judges feel involved.
That said, there needs to be a consistent party line, and the CA should probably ‘chair’ the event and be prepared to give the definitive answer when one is needed.
Use of filmed debates[]
There’s no substitute for seeing a debate, and any training day should include at least one full debate (maybe two), to allow first-time judges to see all the theory in action and to refresh the memories of the more experienced.
In many years a video debate has been shown and then marked out loud by a handful of experienced judges, to give everyone a sense of what sort of marks are appropriate. This is very useful indeed – but shouldn’t degenerate into a discussion between those judges as to whether a particular speech was a 73 or a 74.
In 2002, 2003 and 2007, the whole group of judges was also asked to fill out a ballot for a video debate. This was particularly useful as it allowed the CA to look out for extreme disparities, people marking outside the mark scheme etc. (Note this was not only restricted to first-time judges – some judges have come to WSDC for years and still misunderstand the mark range.)
An idea that has been mooted is that a test ballot should include a small box, for the answer to a question like “What was the most important issue for you in deciding this debate?” This would allow the CA to look out for people awarding debates on very unfair issues, such as fashion sense.
As a rule, though, judges shouldn’t be made to feel terrified by such an exercise, and reassured that as long as their marks and reasons are sensible, they won’t be penalised.
It’s also a CA’s choice whether to show an example video debate that is close or open-and-shut, and whether it features two very experienced teams, two weaker teams, etc.
In 2013 the CAP chose to use a "live" debate comprising student observers accompanying the various delegations. To avoid prejudicing the adjudicators, the nations of the debaters were not revealed. The advantage of this is that adjudicators get to see speeches from a diversity of debating styles and traditions. Many also feel that certain elements of style and persuasion cannot be fully captured in a video. However, the unpredictability of the debate is a potential drawback.
See also[]
- Discussion page for more on the use of marks from a video debate.
Audience interaction[]
It’s good to break up lectures with Q & A and to involve people in the audience as well – who might be a source of good answers as well as questions. Some people do like to contribute more than others, though, and it’s the CA’s job not to let the discussion get dominated by experienced judges – trying to encourage questions from first-timers is more important.
Assessment and accreditation[]
Whether or not a video debate is marked by everyone, the briefing day is the first real opportunity for you and the CAP to learn about new judges, and assess whether they fully understand the requirements for judges. The CAP should meet at the end of the session to look through the ballots (if used) or compare notes on personal impressions.
In addition, it has become common practice to set a short quiz (comprising about 20 short-answer questions) for every adjudicator, covering various aspects of the WSDC format. Typically, material for the quiz has been drawn from the Stockley documents. The results from the quiz can then be used in conjunction with the marksheets from the example debate to assess judges. In general, adjudicators have been required to complete the quiz before the formal briefing, in order to sieve out those who have prior familiarity with the rules.
To prevent bias, both the quiz and the example debate marksheet might be graded annonynmously by the CAP. Instead, each adjudicator is assigned an alpha-numeric code at the start of the briefing. The results are then transcribed once the CAP has finishied assessing the completed quizzes and ballots.
Getting to know the judges[]
As CA, you will want to get to know the judges and get to know as much about them as possible, as soon as possible. Don’t waste this opportunity – the training day is the best opportunity to introduce yourself over a cup of coffee.
The members of the CAP are the ‘face’ of the tournament for judges, more than the Convenor. Remember, some judges may not be close to their team delegations, and may be travelling alone – so the CA and CAP may have a part to play in making sure they enjoy the competition!
Local judges[]
There may be a large pool of local judges willing and able to augment your pool – often teachers responsible for debating at the schools hosting debates or alumni of the host country’s team.
These judges may not be staying in the tournament hotel, where the adjudicator training often takes place, but they should be warmly encouraged to attend that session. However, room space is also often at a premium, so it may be preferable to organise a separate training day a few days earlier for local judges.
It is critical, however, that local judges are taken through the Rules, Judges’ Notes and Definition Guidelines, even if they are unlikely to judge frequently. If they are first-time judges, you might want them to act as shadow judges at first as well (see below).
Judges helping teams[]
One Rule about which judges and others always need reminding: judges are not allowed to help in training teams at any point during the competition. They may do so at home prior to the tournament but it must stop once the competition starts. This does not mean that judges should refuse to adjudicate “spar rounds” which are informal or practice debates frequently organised by different teams prior to the competition, as these spar rounds will (or should) involve constructive criticism to both teams. However, once the competition begins, there should be no more spar rounds and no assistance to teams other than the adjudication comments provided at the end of each debate. This rule is frequently transgressed and it should be pointed out that the CA and CAP will be looking out for clues.
Feedback after the training session[]
We’ve never done this before, other than through anecdotal evidence. However some members of the Executive Committee's Adjudication working group have proposed it. Future CAs might like to apply the techniques of conference management to the training day and ask judges to fill out a form saying what they liked and disliked about the training day/session. Usually the decisions about the structure of the day are taken by those who have sat through several, and not those who were there for the first time. That said, first time judges don’t always know what we’re expecting them to take away from the day – which is a generally complete understanding of the principles of World Schools judging!