How to Review an Official

Process and examples for how to provide feedback about an official

HOW TO REVIEW AN OFFICIAL

This page explains how to provide a review of an official’s performance, be it an overview of their skill for the last several months or years (an “Overview of Officiating Skill”), an evaluation of a single game or tournament (an “Evaluation”), or to provide feedback on that official’s character (“Feedback On Character”).

Jump directly to those sections:

HOW TO COMPLETE AN OVERVIEW OF OFFICIATING SKILL (OOS)

An Overview of Officiating Skill (OOS) is a long-form evaluation designed to capture an official’s performance over the prior two years. These are the most important evaluations of an official’s performance and is often the primary source of evidence for officials applying for Level 1 certification. Reviewers should be the people the Applicant has worked with the most, and who best understands the Applicant’s officiating.

Write an OOS for a Skating Official (Referee)

Write an OOS for a Non-Skating Official (NSO)

NOTE: If you would like to fill out an OOS in a language other than English, you can! Email certification@wftda.com in advance and we will try to find a translator.

Definitions:

Applicant: The person applying for certification

Reviewer: The person completing the OOS

WFTDA: Women’s Flat Track Derby Association

JRDA: Juniors Roller Derby Association

MRDA: Men’s Roller Derby Association

Experienced Official: Any official who meets at least one of the following criteria:

  • Worked a WFTDA Continental Cup, Playoff, or Championship tournament;
  • Worked as tournament Head of an NGB tournament; or
  • Is certified at Level 3.

Although Overviews of Officiating Skill do not expire, we want the most up-to-date picture of the Applicant as possible. For that reason, we ask that the Applicant should not request their OOSes to be filled out until they are prepared to apply (e.g., they should have already worked the suggested number of games, and should either have already completed the education and testing, or plan to do in the following week or two). Unlike evaluations, OOSes are confidential to Certification, though a Reviewer may choose to share what they wrote in their OOS with the Applicant. The Applicant will only see compiled feedback from these in the overall Certification Review Summary.

Each level of Certification requires a different number of and different types of OOSes. You can find out what you need for each level by visiting What do Certification Levels Mean?, but Certification requires the following OOSes to be written by different people:

  • A Mentor or Head Official. For Recognized or Level 1, this is the Applicant’s primary mentor or, if affiliated with a league, could also be the league’s Head Official. For Level 2 or Level 3, this should be written by someone who has observed the Applicant for more advanced sanctioned or regulation games, such as a frequent Crew Head Official, a Tournament Head Official, or a Head Official for many games and scrimmages.
  • Any additional Official. The individual best suited for providing this assessment is a “peer” official, or an official that has worked alongside the Applicant during a significant number of games or scrimmages. It need not be someone who is the “crew head.” If the Applicant is usually the Head Official, they should choose an official who works often on their crews, or a second mentor.
  • Someone writing on behalf of the Skaters: The individual best suited for providing this assessment is a skater who the Applicant has officiated over the past two years, or the period of time over which the Applicant has been officiating prior to seeking certification. The skater doesn’t need to be from the Applicant’s home league, or from a WFTDA, JRDA, or MRDA member league. A league representative can also provide this reference, so long as they are representing the athletes’ perspective (not an administrative perspective, or a third officiating perspective).
  • An Experienced Official (Required for Level 3 Only). The individual best suited to submit this OOS is someone who would otherwise be qualified to fill out an OOS as a Head Official or an additional official, and also has broad experience in the world of roller derby officiating (per the definition above).

OOSes and Evaluations Work Together

  • OOSes provide a broad overview of the official, and should be written by someone who knows their story and has observed their development. OOSes alone are usually enough to certify someone at Level 1.
  • Evaluations provide the minutiae or game-by-game context that illustrates how and when the official excels, and notes how they handle rare situations and edge cases over time, for individual games or tournaments, when the Applicant’s performance is “fresh” in the Reviewer’s mind.
  • Higher levels can be granted via OOSes alone, if the OOSes contain anecdotes and descriptions that serve the same purpose as Evaluations. However, most Reviewers have difficulty remember specific scenarios like this if it has been a long time since they occurred.
  • It is fine for the same person to write one or more Evaluations, as well as an OOS, but one person can only write one OOS.
  • Additional OOSes will be considered but are not recommended. Additional OOSes should only be submitted if the Applicant believes that there will be meaningful context missing from the original OOSes. It is recommended that others use Evaluations to expand upon the information available in a packet rather than additional OOSes.

Identifying Appropriate Reviewers

All of the above categories are designed to be very broad. As a general rule, the Applicant should only ask people who can provide meaningful responses to all of the questions.  Similarly, the Reviewer should only agree to write an OOS if they believe they can provide meaningful perspective on the questions for the Applicant. Notably, if we do not have sufficient information to certify the Applicant, we will note this in their summary, and they might conclude that the Reviewer did not fulfill the role expected.

The Applicant should think about which Reviewers can provide a robust and comprehensive overview of their performance, in multiple roles over multiple events, games or during an extensive period of time. Applicants for Level 2 or Level 3 should ideally include evaluations from multiple leagues and locations in order to demonstrate a regional (Level 2) or global (Level 3) understanding of play.

Applicants who do not have Reviewers who can fit the above categories need to supplement with evaluations so that together the documents can tell a similar story. We recommend evaluations from 5 different games from people filling a similar role to the missing OOS. Here are a few examples:

  • Evaluations from 5 different games from the game or tournament’s Crew Head can replace the Head Official Category;
  • Evaluations from 5 different games from peer officials can replace the Any Other Official Category.

Guidance for Reviewers

An OOS is designed to take about an hour to complete, but could take longer. It includes the following questions:

Reviewer Name and Email Address.
This is to ensure that the person submitting the evaluation is authorized to do so, to send a receipt confirmation and to follow up in case of a future appeal.

How long have you been involved in roller derby, and in which roles?
Please be as clear and concise as possible. We want to understand the level, volume and type of experience you have, but don’t need to see your CV or Game History.

Helpful example:
I’ve been officiating for 5 years, have done around 100 games, mostly within Western Australia. I’m my league’s Head Referee.
Less helpful example:
I’ve been officiating for 5 years.

What category does this OOS fall into?
See the above categories of individuals who may submit an OOS. Reviewers should check with the Applicant to ensure that they select the correct role needed for the certification application, as individuals may qualify for more than one.

Does the Applicant have an Officiating Identity?
All Certified Officials who have registered with Certification are listed in a dropdown menu. Please select the Applicant from the list. If they are not listed, they need to register before you may complete their OOS.

How long have you known the Applicant?
Please explain how and how long you have known the Applicant and how often you have worked together.

Helpful example:
I’ve worked with Bob on 3 occasions as a Tournament Head when they were a crew head. I’ve also attended multiple scrimmages with them in the last 10 years.
Less helpful example:
I’ve known them for 10 years

Do you have a relationship with the Applicant outside of Roller Derby?
For example, if you are personal friends, are romantically involved, or have any other relationship beyond officiating.

Position Specific Performance

After the above questions, the Reviewer will be asked to fill in all the information they have about the Applicant’s performance in specific positions. There are many things to note, including the four axes of officiating performance (further explained below): Skill, Support (of others), Mentorship (of others), and Procedure & Temperament. For these, we are interested in your direct and specific observations.

You don’t need to fill something out for every position, but if you have trouble covering these axes for more than half of the role-specific positions, please discuss it with the Applicant to make sure you are the best reviewer for the Applicant. You may also want to let the Applicant know which roles you cannot provide much info for, so they can ask other reviewers to fill in those gaps.

These areas are the same as those required in an evaluation, and the same kind of detail is helpful.  You may find the guidance on completing an eval helpful. Unlike an evaluation, which is designed to be a snapshot of performance at a single event, you can also reflect how the Applicant has developed in this position over a period of time.

SKILL

All Officials
  • Rules knowledge, including which actions are legal/illegal
  • Attention to detail for the role
  • Ability to handle ambiguous or surprising gameplay scenarios (making a good decision when there is no clear “correct” decision)
Skating Officials Only
  • Skating skill, and if skating ever distracts the Applicant or other referees on the track
  • Ability to identify who initiates actions
  • Knowing the impact of illegal actions
NSOs Only
  • Accuracy of paperwork
  • Performance specific to the role, including correct math and accuracy of timing
  • Attention to detail for the role, including ability to focus and remain engaged in the game

SUPPORT

  • If the Applicant knows the standard practices and tournament-specific practices of the roles
  • Interactions with other officials both on and off skates, specifically their ability to partner or work collaboratively with a crew
  • Interactions with skaters, managers, and other game/tournament staff, including clarity and volume of penalty calls

MENTORSHIP

  • Mentorship of others who are learning the role;
  • Ability to train others in the role
  • Ability to support other officials working on this specific role, even when working in a different role

PROCEDURE AND TEMPERAMENT

  • Descriptions of situations where the Applicant seemed stressed or frustrated in this role
  • Any times the Applicant avoided or forgot the standard practices
  • Any adaptations or improvements the Applicant made in procedures for this role, which may have helped them or the game
  • Ability to adapt to a crew head or tournament head’s request, especially when a process or practice is requested that the Applicant does not usually perform
  • Any patterns you have seen, including scenarios where the Applicant:
    • Commits errors
    • Becomes confused
    • Gets distracted from their role
    • “Takes over” or begins fulfilling other officials’ roles or responsibilities in a manner that is not “support,” such as making out-of-position calls.

Additional Information

Finally, there is a section on officiating expertise outside of specific roles:

What sort of demeanor did the Applicant display before, during and after events?
Explain how the Applicant behaves in the run up to an event (particularly relevant for Applicants working in leadership roles), during an event (not necessarily around their specific role), and after the event (for example, if they are responsible for completing the stats book, or providing feedback). Please be specific, rather than speaking in generalizations.

Helpful example 1 (leadership):

Before events where Katie is a crew head or tournament head (I have worked with her in this capacity on 8 occasions) she is always very clear about what is expected in crew selection, confirms local selection requirements and is collaborative and effective in her engagement with hosts, other officials and ensures rapid, fair and effective crew selections.

At a recent event where I was tournament head and Katie was crew head, she sometimes struggled to manage her time between games, with meetings, feedback and a need to rest and refuel which meant she became frustrated. We discussed this on day one and decided that crew heads wouldn’t attend the captains’ meetings as this significantly impacted on their time between games and meant they were less able to provide feedback and support their crew. This resulted in a better rested, better performing crew on day two.

Why it’s helpful: It’s specific and speaks to the things which were great and those where the official struggled, how it was raised and how the official responded. If there was no discussion of the issue, that should be noted as such.

Helpful example 2:

Two weeks before our games, João let me know that they hadn’t done the specific role before and proactively asked for support in ensuring they were ready. I provided some information, they practiced at scrimmages, and asked follow up questions about game specific procedures.

After games, they often volunteer to complete and submit the statsbook, and do this accurately and in good time.

Why it’s helpful: It’s clear, specific and provides useful triangulation about the official’s skill, performance and knowledge of procedures.

Less helpful example:

Frederica is always pleasant before, during and after events. However, she is disorganized and is sometimes late.

Why it’s less helpful: Although this raises some concerns, it isn’t specific and doesn’t explain the impact of this concern. It is very different to be late to a game (delaying its start) or to be late to a meeting (wasting other officials’ time) than to have been late for “call” due to needing to rely on public transit, but is always on time for everything else.

Are there any situations in which the Applicant performs above and beyond your personal expectations?
This is an area where you can really emphasize the amazing things about the way this Applicant performs. This could be about any aspect of their skill, support, mentorship, temperament or procedures.These examples should be specific, rather than general. It’s OK to leave this section blank, but it’s your opportunity to really explain what this official shines at.

Helpful example:

Tara really shines in a mentorship and officiating development role. After the pandemic nearly all of the local experienced officials retired, and it was proving hard to recruit officials to run safe, effective game days. Tara worked with a number of local leagues to run officials specific bootcamps and recruitment events and facilitated “come and try” sessions at regular scrimmage nights. She has then provided additional training and mentoring to a small group, the first of which has now achieved level 1 certification.

Why it’s helpful: It’s specific, speaks to a clear area of performance and is an area where the person has genuinely gone above and beyond.

Are there any situations that appear to challenge the Applicant? How does the Applicant perform when personally challenged or stressed?

Helpful example:

As OPR, Rainbow Sprite doesn’t do a great job of finding “missing pieces” of information from other officials. For example, in a recent game, they saw a Jammer get hit by a blocker, but didn’t see the Jammer touch out. The rear OPR did, so together this was a cut — the team called an official review and we figured this out, but Sprite could have just asked the Rear OPR “did you see the Jammer touch out?” and sorted it without costing the team their Official Review. Other situations where “missing parts mean silence, instead of asking” include out of play blocks as JR. This is probably because they rarely practice with a full crew, so there’s nobody to ask, but when there is, they need to.

Why it’s helpful: It describes an overall pattern and one possible cause. It also communicates that the Applicant is under-calling rather than over-calling.

Less helpful example:

Because Alia generally conducts herself with an air of efficiency and seriousness during high pressure situations, there may be situations where some may perceive her as being very blunt, and where someone not knowing her well may find them less approachable as a result.

Why it’s less helpful: It highlights an area of challenge, which is good, but does not explain the impact, or what the Applicant has done to address the challenge. It also doesn’t really answer the question.<

Describe a time when you provided feedback to the Applicant regarding a struggle, or a time when you observed the Applicant receiving feedback. How did they receive feedback? How did they adapt based on that feedback?

Helpful example 1:

I have seen Jay receive feedback and apply it immediately, which is the default situation. I have also seen him apply feedback and figure out that it wasn’t working for him, or that it was too much effort for too little reward. In that case, he has gone back to his CH to talk to them about it, but if they insisted, he kept applying the feedback for the tournament, but dropped it afterwards.

In short: he will  adapt when asked to, but if something doesn’t work for him, he will not keep at it beyond the crew in which it was requested. He will however absolutely keep applying any feedback that improves his performance, or even just the parts that work.

To give you a concrete example, I have seen him have to work with a CHR/IPR whose pack definition was, objectively, a little too loose. When given the predictable feedback, by this person, that his own pack definition was too tight, Jay did not argue, but simply loosened his pack definition so that

they would be using the same metrics. He went back to the tighter, correct pack definition after this event.

This past season, he has received very little feedback. It is the problem with being one of the most skilled officials in the region, and subject to very little opportunity for regional travel due to the impact of covid-19.

Why it’s helpful: It’s detailed and specific, talks through the official’s approach to feedback, and recognizes the limitations of the reviewer.

Helpful example 2:

Not necessarily a struggle but whilst I was front IPR we did need to have a conversation around positioning as we both kept tripping over each other and I kept blocking the scoreboard from their view at the jam start. They made the feedback cycle mutual and we both adapted to meet each other’s needs. It was a great way to handle the situation, driven by Patsy.

Why it’s helpful: although it’s short, it’s detailed and specific.

Less helpful example:

They struggle to get feedback, and usually don’t listen, although sometimes they will make a change to make someone happy if they respect them.

Why it’s less helpful: it isn’t detailed, doesn’t provide examples and adds to potential inconsistencies.

Has the Applicant demonstrated the ability to organize, communicate, and execute plans? Include examples.

Helpful example:

William set up the crews for this game even though he was not going to be the Head Referee. He sent me detailed rationale for why people were staffed in these roles, including noting one ref who needed help and support because they were very new. This let me run the crew very effectively. Without this careful organizational set-up, it would have taken me a few jams to detect it and the teams would have been really frustrated!

Why it’s helpful: It describes what the official did, how it was helpful, and why it mattered.

Less helpful example:

William was a really good head referee for the host league, and thanks to that he set me up to do good as Head Referee for the game!

Why it’s less helpful: It does not describe what it means to be a “good” head referee, or how he set up the Reviewer.

When in a leadership position during a game, how did the Applicant give feedback to other Officials?
If you have not observed this official in a leadership role, you may leave this question blank; leadership roles include being Crew Head, but may include other positions as well, especially if paired with newer officials in order to assist them while completing a role.

Helpful example:

Priscilla did a great job as a Head Referee and gave us feedback during the game in small easy-to-digest bits. She listened to the Alt Ref and gave us good context about how we should take and apply their feedback. This was important because the Alt Ref was pretty new and some of the feedback wasn’t very good — this let us know where we stood. We did miss a big cut on the outside though and Priscilla told us that it was really bad and that we can’t let it happen again, which wasn’t very actionable and left everyone feeling pretty embarrassed and bad about their performance. But that only happened in one game for the whole tournament so overall it was a good experience.

Why it’s helpful: It contains specific examples and describes how and why their feedback style can be improved, and when it works.

Less helpful example:

Priscilla gives great feedback to us during games, I always have something to learn from her.

Why it’s less helpful: The lack of examples or description of the process means we don’t know if Priscilla’s style would work for everybody, or if it just works for this one Reviewer. Every official takes feedback differently, so we need to know what is the Reviewer’s personal experience but also how it might apply to others.

How effectively did the Applicant communicate with the organizers, teams, and other Officials?
Many officials, even when not in a Head Official capacity, may interact with others in larger settings.

Helpful example:

As a crew head Katie’s communication with their crew and myself as a tournament head was clear and helpful. English was an additional language for all of her crew, so she translated information into their (shared) first language wherever possible to ensure clarity of messaging from organizers and tournament heads

Katie also offered to hold official reviews in French as both teams were French, which enabled a smoother flow of gameplay, without additional delays. She was the PBM and also the HNSO for this game and one question was specifically about a Jammer Swap, but she still translated it for everyone to hear and understand even though she could have just answered in French and moved on.

Why it’s helpful: this is very specific, and provides great evidence. It may be necessary for the panel to look slightly wider to gain an understanding about whether this is a one-off or part of a pattern

Anything else you would like to share?
We are interested in your additional thoughts.

Would you want to work with this Applicant again? Please rate the extent to which you would want future games to include this Applicant in this role.
On a scale of 1 (Definitely Not) to 5 (Definitely) the assessor is asked to rate the extent to which they would want the official to be staffed in the assessed role(s) in future games. This information is not shared with the official.

Sample Evaluation

Shared Performance Analysis (This is ALWAYS shared with the Official)

Describe this Official’s in-game performance, and how the performance affected the game in question. This information will be shared with the Official being evaluated. You can write as much as you want here, even though it looks like it’s just one line.

While describing performance, you should consider the aspects of the review section. This includes the following areas:

  • Skill: knowledge and attention to role
  • Support: standard practices, interactions with other officials
  • Mentorship: ability to support others in this or other roles
  • Procedure and Temperament: situations where the official seemed stressed, avoided standard practices, adaptations to improve the game, response to feedback
  • For Skating Officials: Skating skill, identification of initiation, impact of action
  • NSOs: Accuracy of paperwork, performance including maths and accuracy of timing, attention to detail and focus

For most games, you may not be able to write about every item listed above, and that is OK. Over time, multiple evals for multiple games, together with Overviews, will paint the full picture of the official. For a game where something stands out about the officials’ performance, please include it. These notes often provide detail and specificity about the official and give evidence required for the panel to advance an official an additional level.

These shared performance evaluations should take ten to thirty minutes to write. We encourage you to write in your native language, and we will translate it. English is not required. If you find it takes longer than thirty minutes to write, and there is nothing “special” about the game, you may be working too hard!

Useful Example Evaluation of a Leadership Official

Sally was a CH/EIWB at this event and did an excellent job from their initial selection and throughout the event. They were able to work in line with the guidelines for crew selection, understood where people were likely to succeed and were flexible in making adjustments. This meant that selection was smooth.

Leading up to the event, they did a great job communicating with the crew, ensuring everyone had all the info they needed and were aware of what was expected.

During the event, they were able to complete the technical aspects of their role with focus, accuracy and attention to detail whilst supporting and mentoring their crew. This meant that everyone operated much better in high-intensity games knowing their CH had their back, had great feedback to offer, but also wanted everyone to have a good time.

They were also able to pass along feedback and info the THs and ALTs provided – accurately conveying the message in actionable ways that crew-members could immediately implement.

They ensured communication across the positional families was consistent throughout each game – star passes, penalty counts, penalties served.

They worked well with their paired CHR – ensuring communication was on point for needs on both sides. There were no clock stoppages for needing to fix things on the NSO side.

Why it’s good: This evaluation speaks to Skill (in role), Support (of other NSOs, the CHR and other officials including leadership), Mentorship (of their crew) and Procedure (specifically around giving and receiving feedback). It provides highly specific examples of how their work had a positive impact, how they communicated and would support decision making for the Panel.

Useful Example Evaluation of a Jam Timer

Kai was Jam Timer at Made-up Tournament, performing 5 games over the weekend. I was their Tournament Head NSO. They performed well throughout some very testing games.

Specifically, they were able to perform their role knowledgeably and with appropriate attention to detail in an extremely loud hall, with direct sunlight across the pivot line at multiple times throughout the day. They were paired with a far less experienced scoreboard operator, with whom they communicated clearly, concisely and non-patronisingly. Their whistles and hand signals were crisp and highly visible and they provided good information to other officials on the infield.

They worked well with the scoreboard operator, who was positioned in turn 2 due to venue constraints, and required few corrections. They provided useful context when needed, such as around official reviews and a complex double official review, one of which was retained, and their knowledge of the scoreboard enabled them to explain how to resolve this quickly and effectively, without impacting game flow. They were also able to effectively catch all communication from bench coaches despite less-than ideal venue conditions.

They followed all processes throughout the weekend and maintained a calm, unruffled exterior during very close, tense and high-level gameplay as part of a very mixed level crew. This was invaluable to the Crew Head NSO, who was therefore able to focus on supporting others.

One jam ran 3 seconds long, due to a momentary loss of focus, however they resolved timing issues with the scoreboard and head officials well, an this did not impact gameplay

Why it’s good: This evaluation speaks to Skill (in role), Support (of other NSOs, particularly the SBO), Mentorship (of the SBO) and Procedure (following game specific, maintaining a calm temperament) and NSO specific (accuracy of timing, including how they recovered from an error). It provides highly specific examples of how their work had a positive impact, how they communicated and would support decision making for the Panel.

Useful Example Evaluation for a Jammer Referee

Skill
Bob came across as reasonably confident and competent in the role, with a few minor etiquette things to work on as well as some footwork to improve positioning. Points were awarded accurately, and they retained focus on their jammer throughout.

Lead appeared to be awarded accurately and promptly. During our discussion afterwards they admitted that he had been slow to call off one time as they were looking at their SK. Whistles were confident and correct. Bob wasn’t touching their hips to signal when their jammer called it off so we discussed this afterwards and why it is important as it communicates to others, including their SK that it was their jammer that called the jam.

Response to Feedback
After the game we had a conversation and we spoke about the things I had noted down and discussed some footwork skills to work on in practice. They were very receptive to this

Skating Skill
Positioning could be improved. We discussed that when the jammer moves back slightly that they should reposition themselves, even if it’s only a short distance. They acknowledged that movements in this direction was something they needed to work on. When the jammer is stationary they will face the track but could do with practicing some footwork to assist with  lateral movements both ways while facing the track because as soon as the jammer moves they are side on again.

General skate skills are satisfactory, they could keep up with the jammers at this level and stop appropriately prior to the jammer reaching the pack.

Why it’s good: This provides detailed, highly specific feedback on a specific role, with reference to Skill (in role), Support (communication with SKs), Temperament (feedback) and Skating Skill. It’s also clearly laid out and will support at a glance identification of areas which may be missing from other evidence

Less-Useful Example Evaluation

He is very reliable and handles the nuances of NSOing competently, keeping games running smoothly.

What could improve: This is not specific to the role being carried out at the event evaluated, or the person being evaluated. It also does not speak to skill, support, mentorship or the specific criteria for the NSO position undertaken.

Broader Feedback (you can choose whether to share this with the Official)

In this section, you can note how performance or conduct shown in this game may have impact beyond the game in question, even if they did not have an effect on this specific game. Certification will look for these broader themes during Certification Review. You can write as much as you want here, even though it looks like it’s just one line.

This section can be used for information which relates to broader themes, or if you have feedback you wish to share which you do not wish to share with the official. To enable this information to be actionable it may be helpful to consider which of the following criteria it falls within:

  • Overall understanding of the role (besides just “how they performed” in this game)
  • Conduct or demeanor, whether during or before/after the game
  • Giving, Receiving, or Actioning on Feedback
  • Organisation/Readiness to fulfill responsibilities
  • Communication (Are they clear? Are they concise? Are they polite and respectful?)
  • Leadership (where applicable)

It is OK for you to not have any broader feedback for an official for a game. If you do have broader feedback, you should be able to write it in fifteen minutes or less.

IMPORTANT NOTE: Evals are not routinely reviewed as they are received. Instead, they are included in an official’s “packet” for when they next apply for certification. If you have a serious concern or allegation relating to a certified official, you should highlight this by email immediately to cert.oversight@wftda.com or via the WFTDA grievance process.

Back to top