While diligent and exhaustive evaluation of a goaltender is the ideal, this is often out of reach of many amateur organizations, and even some professional ones. The simple truth is that scouts, coaches, and evaluators do not have the time or resources to watch multiple games, skills sessions, combines, etc., for every goaltender they come across. Therefore, organizations need to make prudent decisions as to who they evaluate, how they evaluate them, and for how long.
In order to make effective resourcing decisions, there are two recommended approaches to simplifying the system detailed above. The first method is to simplify and reduce the evaluation criteria, and the second is to select the most efficient evaluation types, or “looks,” that will provide the most information for the time spent. Both may be used in tandem or independently.
The rating system in this methodology encourages users to leave criteria blank and only rate those traits which are evident in the goaltender’s performance. However, for those without extensive goaltending knowledge, the criteria may still be difficult to understand. Similarly, in a high-volume skills session where tens of goaltenders are being evaluated (e.g. District and Regional Selection Camps), there may only be enough time to observe a limited number of repetitions per goalie, therefore requiring snap decisions on the part of evaluators.
In these situations, it makes sense to scale back the criteria to the higher-level categories within the system:
Those lacking position-specific knowledge should still be able to evaluate according to higher-level definitions of these categories. For example, they should be able to define how well a goaltender moves, how consistently they are able to make a save, whether or not they are able to read the play, etc., especially when deciding how these categories factor into the goaltender’s ability to compete. Similarly, in high-volume situations, an evaluator should be able to quickly decide on category scoring without having to determine root cause for a specific success or failure.
For resource-strapped organizations, the other means of scaling is by choosing those “looks,” or evaluation types, that get the most “bang for your buck” in terms of time spent. The graphic below shows several examples of evaluation type along with the specific categories that are ideally evaluated in those looks:
It should not be surprising that observing games is the ideal method to assess all of a goaltender’s traits. While it may be somewhat difficult to assess Learning and Growth Habits, there are still observable behaviors that may indicate some of those traits. Another valuable option is to gather first-hand feedback from team coaches, educators, and/or service providers who work directly with the athlete. For youth organizations who select internally, this could be annual coaches’ feedback; and for organizations who scout, this could take the form of references, though these will usually be taken with a grain of salt.
Oftentimes professional and high performance amateur organizations will also look at a goaltender’s statistics and factor in the level of competition faced. Evaluators may also make assumptions based on the development environment that the athlete came from. For example, if the athlete’s organization is reputable with a high proportion of NHL alumni, they may be given higher preference over an athlete from a less prestigious organization.
While elements like statistics and background somewhat indicate the efficacy of a goaltender’s skillset, the evaluation is less about the goaltender themselves and more about their specific environment. In the case of a development audit, the evaluation looks at the impact from the goaltender’s environment. Inversely, a statistical analysis shows the impact of the goaltender on their environment. In a vacuum, these methods very rarely shed light on a goaltender’s ability to process, decide, and act within those environments — it is just an assessment of the cycle’s input and output.
As organizations narrow down to a short list of athletes, there is often a willingness to spend further resources evaluating these finalists. In these circumstances, it may be worth viewing a skills session or inviting the athlete to a training camp, conducting an interview or personality assessment, or doing athletic testing. While this supplemental knowledge can be helpful for determining a development pathway, evaluators should still be cognizant of the limitations and should not deduce behaviors that cannot be adequately assessed. It is perfectly acceptable to leave criteria blank.