How to Review an Official
Who Should Review an Official?
HOW TO COMPLETE AN OVERVIEW OF OFFICIATING SKILL (OOS)
An Overview of Officiating Skill (OOS) is a long-form evaluation designed to capture an official’s performance over the prior two years. These are the most important evaluations of an official’s performance and is often the primary source of evidence for officials applying for Level 1 certification. Reviewers should be the people the Applicant has worked with the most, and who best understands the Applicant’s officiating.
Write an OOS for a Skating Official (Referee)
Write an OOS for a Non-Skating Official (NSO)
NOTE: If you would like to fill out an OOS in a language other than English, you can! Email certification@wftda.com in advance and we will try to find a translator.
Definitions:
Applicant: The person applying for certification
Reviewer: The person completing the OOS
WFTDA: Women’s Flat Track Derby Association
JRDA: Juniors Roller Derby Association
MRDA: Men’s Roller Derby Association
Experienced Official: Any official who meets at least one of the following criteria:
- Worked a WFTDA Continental Cup, Playoff, or Championship tournament;
- Worked as tournament Head of an NGB tournament; or
- Is certified at Level 3.
Although Overviews of Officiating Skill do not expire, we want the most up-to-date picture of the Applicant as possible. For that reason, we ask that the Applicant should not request their OOSes to be filled out until they are prepared to apply (e.g., they should have already worked the suggested number of games, and should either have already completed the education and testing, or plan to do in the following week or two). Unlike evaluations, OOSes are confidential to Certification, though a Reviewer may choose to share what they wrote in their OOS with the Applicant. The Applicant will only see compiled feedback from these in the overall Certification Review Summary.
Each level of Certification requires a different number of and different types of OOSes. You can find out what you need for each level by visiting What do Certification Levels Mean?, but Certification requires the following OOSes to be written by different people:
- A Mentor or Head Official. For Recognized or Level 1, this is the Applicant’s primary mentor or, if affiliated with a league, could also be the league’s Head Official. For Level 2 or Level 3, this should be written by someone who has observed the Applicant for more advanced sanctioned or regulation games, such as a frequent Crew Head Official, a Tournament Head Official, or a Head Official for many games and scrimmages.
- Any additional Official. The individual best suited for providing this assessment is a “peer” official, or an official that has worked alongside the Applicant during a significant number of games or scrimmages. It need not be someone who is the “crew head.” If the Applicant is usually the Head Official, they should choose an official who works often on their crews, or a second mentor.
- Someone writing on behalf of the Skaters: The individual best suited for providing this assessment is a skater who the Applicant has officiated over the past two years, or the period of time over which the Applicant has been officiating prior to seeking certification. The skater doesn’t need to be from the Applicant’s home league, or from a WFTDA, JRDA, or MRDA member league. A league representative can also provide this reference, so long as they are representing the athletes’ perspective (not an administrative perspective, or a third officiating perspective).
- An Experienced Official (Required for Level 3 Only). The individual best suited to submit this OOS is someone who would otherwise be qualified to fill out an OOS as a Head Official or an additional official, and also has broad experience in the world of roller derby officiating (per the definition above).
OOSes and Evaluations Work Together
- OOSes provide a broad overview of the official, and should be written by someone who knows their story and has observed their development. OOSes alone are usually enough to certify someone at Level 1.
- Evaluations provide the minutiae or game-by-game context that illustrates how and when the official excels, and notes how they handle rare situations and edge cases over time, for individual games or tournaments, when the Applicant’s performance is “fresh” in the Reviewer’s mind.
- Higher levels can be granted via OOSes alone, if the OOSes contain anecdotes and descriptions that serve the same purpose as Evaluations. However, most Reviewers have difficulty remember specific scenarios like this if it has been a long time since they occurred.
- It is fine for the same person to write one or more Evaluations, as well as an OOS, but one person can only write one OOS.
- Additional OOSes will be considered but are not recommended. Additional OOSes should only be submitted if the Applicant believes that there will be meaningful context missing from the original OOSes. It is recommended that others use Evaluations to expand upon the information available in a packet rather than additional OOSes.
Identifying Appropriate Reviewers
All of the above categories are designed to be very broad. As a general rule, the Applicant should only ask people who can provide meaningful responses to all of the questions. Similarly, the Reviewer should only agree to write an OOS if they believe they can provide meaningful perspective on the questions for the Applicant. Notably, if we do not have sufficient information to certify the Applicant, we will note this in their summary, and they might conclude that the Reviewer did not fulfill the role expected.
The Applicant should think about which Reviewers can provide a robust and comprehensive overview of their performance, in multiple roles over multiple events, games or during an extensive period of time. Applicants for Level 2 or Level 3 should ideally include evaluations from multiple leagues and locations in order to demonstrate a regional (Level 2) or global (Level 3) understanding of play.
Applicants who do not have Reviewers who can fit the above categories need to supplement with evaluations so that together the documents can tell a similar story. We recommend evaluations from 5 different games from people filling a similar role to the missing OOS. Here are a few examples:
- Evaluations from 5 different games from the game or tournament’s Crew Head can replace the Head Official Category;
- Evaluations from 5 different games from peer officials can replace the Any Other Official Category.
Guidance for Reviewers
An OOS is designed to take about an hour to complete, but could take longer. It includes the following questions:
Reviewer Name and Email Address.
This is to ensure that the person submitting the evaluation is authorized to do so, to send a receipt confirmation and to follow up in case of a future appeal.
How long have you been involved in roller derby, and in which roles?
Please be as clear and concise as possible. We want to understand the level, volume and type of experience you have, but don’t need to see your CV or Game History.
I’ve been officiating for 5 years, have done around 100 games, mostly within Western Australia. I’m my league’s Head Referee.
I’ve been officiating for 5 years.
What category does this OOS fall into?
See the above categories of individuals who may submit an OOS. Reviewers should check with the Applicant to ensure that they select the correct role needed for the certification application, as individuals may qualify for more than one.
Does the Applicant have an Officiating Identity?
All Certified Officials who have registered with Certification are listed in a dropdown menu. Please select the Applicant from the list. If they are not listed, they need to register before you may complete their OOS.
How long have you known the Applicant?
Please explain how and how long you have known the Applicant and how often you have worked together.
I’ve worked with Bob on 3 occasions as a Tournament Head when they were a crew head. I’ve also attended multiple scrimmages with them in the last 10 years.
I’ve known them for 10 years
Do you have a relationship with the Applicant outside of Roller Derby?
For example, if you are personal friends, are romantically involved, or have any other relationship beyond officiating.
Position Specific Performance
After the above questions, the Reviewer will be asked to fill in all the information they have about the Applicant’s performance in specific positions. There are many things to note, including the four axes of officiating performance (further explained below): Skill, Support (of others), Mentorship (of others), and Procedure & Temperament. For these, we are interested in your direct and specific observations.
You don’t need to fill something out for every position, but if you have trouble covering these axes for more than half of the role-specific positions, please discuss it with the Applicant to make sure you are the best reviewer for the Applicant. You may also want to let the Applicant know which roles you cannot provide much info for, so they can ask other reviewers to fill in those gaps.
These areas are the same as those required in an evaluation, and the same kind of detail is helpful. You may find the guidance on completing an eval helpful. Unlike an evaluation, which is designed to be a snapshot of performance at a single event, you can also reflect how the Applicant has developed in this position over a period of time.
SKILL
All Officials
- Rules knowledge, including which actions are legal/illegal
- Attention to detail for the role
- Ability to handle ambiguous or surprising gameplay scenarios (making a good decision when there is no clear “correct” decision)
Skating Officials Only
- Skating skill, and if skating ever distracts the Applicant or other referees on the track
- Ability to identify who initiates actions
- Knowing the impact of illegal actions
NSOs Only
- Accuracy of paperwork
- Performance specific to the role, including correct math and accuracy of timing
- Attention to detail for the role, including ability to focus and remain engaged in the game
SUPPORT
- If the Applicant knows the standard practices and tournament-specific practices of the roles
- Interactions with other officials both on and off skates, specifically their ability to partner or work collaboratively with a crew
- Interactions with skaters, managers, and other game/tournament staff, including clarity and volume of penalty calls
MENTORSHIP
- Mentorship of others who are learning the role;
- Ability to train others in the role
- Ability to support other officials working on this specific role, even when working in a different role
PROCEDURE AND TEMPERAMENT
- Descriptions of situations where the Applicant seemed stressed or frustrated in this role
- Any times the Applicant avoided or forgot the standard practices
- Any adaptations or improvements the Applicant made in procedures for this role, which may have helped them or the game
- Ability to adapt to a crew head or tournament head’s request, especially when a process or practice is requested that the Applicant does not usually perform
- Any patterns you have seen, including scenarios where the Applicant:
- Commits errors
- Becomes confused
- Gets distracted from their role
- “Takes over” or begins fulfilling other officials’ roles or responsibilities in a manner that is not “support,” such as making out-of-position calls.
Additional Information
Finally, there is a section on officiating expertise outside of specific roles:
What sort of demeanor did the Applicant display before, during and after events?
Explain how the Applicant behaves in the run up to an event (particularly relevant for Applicants working in leadership roles), during an event (not necessarily around their specific role), and after the event (for example, if they are responsible for completing the stats book, or providing feedback). Please be specific, rather than speaking in generalizations.
Helpful example 1 (leadership):
Before events where Katie is a crew head or tournament head (I have worked with her in this capacity on 8 occasions) she is always very clear about what is expected in crew selection, confirms local selection requirements and is collaborative and effective in her engagement with hosts, other officials and ensures rapid, fair and effective crew selections.
At a recent event where I was tournament head and Katie was crew head, she sometimes struggled to manage her time between games, with meetings, feedback and a need to rest and refuel which meant she became frustrated. We discussed this on day one and decided that crew heads wouldn’t attend the captains’ meetings as this significantly impacted on their time between games and meant they were less able to provide feedback and support their crew. This resulted in a better rested, better performing crew on day two.
Why it’s helpful: It’s specific and speaks to the things which were great and those where the official struggled, how it was raised and how the official responded. If there was no discussion of the issue, that should be noted as such.
Helpful example 2:
Two weeks before our games, João let me know that they hadn’t done the specific role before and proactively asked for support in ensuring they were ready. I provided some information, they practiced at scrimmages, and asked follow up questions about game specific procedures.
After games, they often volunteer to complete and submit the statsbook, and do this accurately and in good time.
Why it’s helpful: It’s clear, specific and provides useful triangulation about the official’s skill, performance and knowledge of procedures.
Less helpful example:
Frederica is always pleasant before, during and after events. However, she is disorganized and is sometimes late.
Why it’s less helpful: Although this raises some concerns, it isn’t specific and doesn’t explain the impact of this concern. It is very different to be late to a game (delaying its start) or to be late to a meeting (wasting other officials’ time) than to have been late for “call” due to needing to rely on public transit, but is always on time for everything else.
Are there any situations in which the Applicant performs above and beyond your personal expectations?
This is an area where you can really emphasize the amazing things about the way this Applicant performs. This could be about any aspect of their skill, support, mentorship, temperament or procedures.These examples should be specific, rather than general. It’s OK to leave this section blank, but it’s your opportunity to really explain what this official shines at.
Helpful example:
Tara really shines in a mentorship and officiating development role. After the pandemic nearly all of the local experienced officials retired, and it was proving hard to recruit officials to run safe, effective game days. Tara worked with a number of local leagues to run officials specific bootcamps and recruitment events and facilitated “come and try” sessions at regular scrimmage nights. She has then provided additional training and mentoring to a small group, the first of which has now achieved level 1 certification.
Why it’s helpful: It’s specific, speaks to a clear area of performance and is an area where the person has genuinely gone above and beyond.
Are there any situations that appear to challenge the Applicant? How does the Applicant perform when personally challenged or stressed?
Helpful example:
As OPR, Rainbow Sprite doesn’t do a great job of finding “missing pieces” of information from other officials. For example, in a recent game, they saw a Jammer get hit by a blocker, but didn’t see the Jammer touch out. The rear OPR did, so together this was a cut — the team called an official review and we figured this out, but Sprite could have just asked the Rear OPR “did you see the Jammer touch out?” and sorted it without costing the team their Official Review. Other situations where “missing parts mean silence, instead of asking” include out of play blocks as JR. This is probably because they rarely practice with a full crew, so there’s nobody to ask, but when there is, they need to.
Why it’s helpful: It describes an overall pattern and one possible cause. It also communicates that the Applicant is under-calling rather than over-calling.
Less helpful example:
Because Alia generally conducts herself with an air of efficiency and seriousness during high pressure situations, there may be situations where some may perceive her as being very blunt, and where someone not knowing her well may find them less approachable as a result.
Why it’s less helpful: It highlights an area of challenge, which is good, but does not explain the impact, or what the Applicant has done to address the challenge. It also doesn’t really answer the question.<
Describe a time when you provided feedback to the Applicant regarding a struggle, or a time when you observed the Applicant receiving feedback. How did they receive feedback? How did they adapt based on that feedback?
Helpful example 1:
I have seen Jay receive feedback and apply it immediately, which is the default situation. I have also seen him apply feedback and figure out that it wasn’t working for him, or that it was too much effort for too little reward. In that case, he has gone back to his CH to talk to them about it, but if they insisted, he kept applying the feedback for the tournament, but dropped it afterwards.
In short: he will adapt when asked to, but if something doesn’t work for him, he will not keep at it beyond the crew in which it was requested. He will however absolutely keep applying any feedback that improves his performance, or even just the parts that work.
To give you a concrete example, I have seen him have to work with a CHR/IPR whose pack definition was, objectively, a little too loose. When given the predictable feedback, by this person, that his own pack definition was too tight, Jay did not argue, but simply loosened his pack definition so that
they would be using the same metrics. He went back to the tighter, correct pack definition after this event.
This past season, he has received very little feedback. It is the problem with being one of the most skilled officials in the region, and subject to very little opportunity for regional travel due to the impact of covid-19.
Why it’s helpful: It’s detailed and specific, talks through the official’s approach to feedback, and recognizes the limitations of the reviewer.
Helpful example 2:
Not necessarily a struggle but whilst I was front IPR we did need to have a conversation around positioning as we both kept tripping over each other and I kept blocking the scoreboard from their view at the jam start. They made the feedback cycle mutual and we both adapted to meet each other’s needs. It was a great way to handle the situation, driven by Patsy.
Why it’s helpful: although it’s short, it’s detailed and specific.
Less helpful example:
They struggle to get feedback, and usually don’t listen, although sometimes they will make a change to make someone happy if they respect them.
Why it’s less helpful: it isn’t detailed, doesn’t provide examples and adds to potential inconsistencies.
Has the Applicant demonstrated the ability to organize, communicate, and execute plans? Include examples.
Helpful example:
William set up the crews for this game even though he was not going to be the Head Referee. He sent me detailed rationale for why people were staffed in these roles, including noting one ref who needed help and support because they were very new. This let me run the crew very effectively. Without this careful organizational set-up, it would have taken me a few jams to detect it and the teams would have been really frustrated!
Why it’s helpful: It describes what the official did, how it was helpful, and why it mattered.
Less helpful example:
William was a really good head referee for the host league, and thanks to that he set me up to do good as Head Referee for the game!
Why it’s less helpful: It does not describe what it means to be a “good” head referee, or how he set up the Reviewer.
When in a leadership position during a game, how did the Applicant give feedback to other Officials?
If you have not observed this official in a leadership role, you may leave this question blank; leadership roles include being Crew Head, but may include other positions as well, especially if paired with newer officials in order to assist them while completing a role.
Helpful example:
Priscilla did a great job as a Head Referee and gave us feedback during the game in small easy-to-digest bits. She listened to the Alt Ref and gave us good context about how we should take and apply their feedback. This was important because the Alt Ref was pretty new and some of the feedback wasn’t very good — this let us know where we stood. We did miss a big cut on the outside though and Priscilla told us that it was really bad and that we can’t let it happen again, which wasn’t very actionable and left everyone feeling pretty embarrassed and bad about their performance. But that only happened in one game for the whole tournament so overall it was a good experience.
Why it’s helpful: It contains specific examples and describes how and why their feedback style can be improved, and when it works.
Less helpful example:
Priscilla gives great feedback to us during games, I always have something to learn from her.
Why it’s less helpful: The lack of examples or description of the process means we don’t know if Priscilla’s style would work for everybody, or if it just works for this one Reviewer. Every official takes feedback differently, so we need to know what is the Reviewer’s personal experience but also how it might apply to others.
How effectively did the Applicant communicate with the organizers, teams, and other Officials?
Many officials, even when not in a Head Official capacity, may interact with others in larger settings.
Helpful example:
As a crew head Katie’s communication with their crew and myself as a tournament head was clear and helpful. English was an additional language for all of her crew, so she translated information into their (shared) first language wherever possible to ensure clarity of messaging from organizers and tournament heads
Katie also offered to hold official reviews in French as both teams were French, which enabled a smoother flow of gameplay, without additional delays. She was the PBM and also the HNSO for this game and one question was specifically about a Jammer Swap, but she still translated it for everyone to hear and understand even though she could have just answered in French and moved on.
Why it’s helpful: this is very specific, and provides great evidence. It may be necessary for the panel to look slightly wider to gain an understanding about whether this is a one-off or part of a pattern
Anything else you would like to share?
We are interested in your additional thoughts.
Would you want to work with this Applicant again? Please rate the extent to which you would want future games to include this Applicant in this role.
On a scale of 1 (Definitely Not) to 5 (Definitely) the assessor is asked to rate the extent to which they would want the official to be staffed in the assessed role(s) in future games. This information is not shared with the official.