Intro
The AUDL tracks more stats than anybody else in Ultimate. What's more is they track these stats completely, i.e. for every player in every game, which makes statistical analysis and learnings a lot more possible than was previously true. These stats have already started to enable new understandings of Ultimate in a few places (one, two). Yardage and field position is particularly useful.
This post is not an indictment of AUDL's state-of-the-art (within Ultimate) statkeeping, but instead a list of suggestions. There are a few ways that the AUDL (along with other Ultimate beancounting organizations) could easily improve their statkeeping. I divide these changes into two categories: directly counted stats, and indirect metrics that are calculated after the fact.
Direct Statkeeping Changes
These are all changes to the ways that statkeepers record stats. None of them are new stats.
Yardage from end destination
Change
Currently, yardage is recorded from the spot where the catch happens. It should be recorded from the place
where the receiver stops running and sets his pivot.
Reasoning
The place where the disc ends up after the receiver stops fully includes all the yardage someone is responsible
for, whether he is throwing or receiving. If you throw someone a so-called fifteen-yard under, by the time the
receiver has come to a stop, it's more like a seven-yard under. It also makes a huge difference whether a receiver's
momentum is towards the sideline or the middle of the field when they catch the disc in the same spot. Statkeeping
should reflect this.
Impact 🥏 🥏 / 5
The only real difference in stats would be among players who throw relatively many or few uplines and hucks compared to
unders and swings. I hypothesize this change to be small as I bet that there is not much variance between
the distribution of types of throws that different players complete.
Yards on Throwaways
Change
The yards you gain (or lose) on a throwaway should be recorded just like any other yards.
Reasoning
When you throw a turn, you gain or lose yards for your team, just like any other throw. If I throw a sixty
yard turnover, then the other team has a zero-yard possession, and then I throw a ten-yard completion, we're in
the same position as if I throw a ten-yard turnover, the other team has a zero-yard possession, and then I throw a
sixty yard completion - why is one situation recorded as one turn and ten throwing yards, and the other is recorded as one
turn and sixty throwing yards? In both cases I have thrown one turnover and advanced the disc seventy yards.
Yards from turnovers are just as legitimate as yards from completions and it's mind-boggling that they're not tracked.
Impact 🥏 🥏 🥏 / 5
Turnovers Forced
Change
Any block that you're responsible for should be recorded, whether or not you touch the disc.
Reasoning
A hand block is the same as forcing the thrower to throw it into the ground. A deep block is the same as forcing
a receiver to go up too early and miss it. Getting a flesh-on-plastic block on the dump is the same as forcing an overthrow on
a swing. The downside here is that this puts more responsibility into the hands of statkeepers who must make
more subjective decisions, but I think that is a worthwhile tradeoff (and if this was a big worry, we could record
disc-touching blocks versus non-disc-touching blocks).
Impact 🥏 🥏 🥏 🥏 🥏/ 5
Getting turnover-causing more correctly recorded would shift our understanding of what is important in defense.
I think there are players who skew dramatically towards forcing turnovers where they don't turn over the disc
and players who skew dramatically the opposite way. Right now, the players who are most overrepresented in
getting blocks are the players who can only get blocks by touching the disc, which makes me think that smarter
players and players who play “lockdown” defense are undervalued.
Calculated Metrics Changes
There are a couple nice-to-haves that would be very easy to incorporate to the site without having to take additional stats.
Remove Plus-Minus
Change
Plus minus should be removed.
Reasoning
Plus minus is a truly insane stat. Why would a goal be equal in value to a block and both equal to
a turnover? It's bonkers on its face and obviously biased (at the level of the AUDL) towards goals and assists and
away from blocks and turnovers. Nobody would ever want a player on their team who scores five goals but throws three
turnovers per game, but someone who gets 5 blocks per game and throws 3 turnovers would be the unanimous league MVP
- yet these players have identical plus minus. Plus minus does nothing but confuse viewers and encourage us to
to ignore stats that contain meaning. It's the opium of the masses. In my opinion plus minus only
exists to satisfy the lazy statistician's appetite to sum up all of performance into one number. I'll probably make
a separate blog post about the failure of plus minus, why it still exists, and how we can fix it.
Impact 🥏 / 5
This change wouldn't add any information. And anyone who knows anything about stats is already ignoring plus minus, anyway.
Confidence Intervals
Change
Every stat should have confidence intervals.
Reasoning
Why should anyone care if some player has a completion percentage higher than someone else if there does not exist a
statistically significant difference between the two numbers? Some reported stats do contain cutoffs as a proxy for
statistical significance (for example, the league often cites stats such as “highest huck completion - minimum 10 hucks”).
Everyone knows that someone with a 100% completion rate but who has only thrown two throws all season should not be
regarded as the most reliable thrower in the AUDL, but can we say for sure, for example, that the thrower who completed 98.84%
of his 172 throws has a better completion rate than the thrower who completed 98.70% of his 77 throws (those are the two
completion percentage leaders for players with a sub-100 percent completion percentage)? The solution is confidence
intervals (or, alternately, p-values when comparing players).
Impact 🥏 🥏 / 5
I would still expect most people to ignore the confidence intervals.
Expected Number of Goals Contributed
Change
Introduce a stat representing how much a player's throwing and receiving changed their team's expected number of goals.
Reasoning
It's relatively straightforward to calculate a reasonable estimate for the expected number of goals per point at
any point on the field (for example, see this webapp). This would only have to be done once.
Then, throws can be understood as changes in expected goals - if you throw a huck from your own endzone to the
doorstep of the other, your percent chance of scoring goes way up; if you throw a swing to the sideline, your percent
chance of scoring goes down; if you throw a goal, your percent chance of scoring becomes 100%, which correctly captures
how valuable scoring is. With the change in expected value for each throw, including turnovers
(your expected goals per point is the inverse of your opponent's same), we can sum up offensive contribution over
a point, game, or season, separating out (as with yards) into throwing and receiving contribution.
One downside is that the estimate of expected number of goals at any point on the field is imperfect - different teams have
different distributions and other factors such as throwing to different players changes the expected value, but narrowing our
dataset to account for these factors likely leaves us with not enough confidence.
This change would take some effort to build compared to the other changes mentioned in this post. But it would be a valuable
model we can use to more thoroughly evaluate players and their contributions.
Impact 🥏 🥏 🥏 / 5
I predict this stat would be quite similar to yards per turn.
Yards Per Turn
Change
Yards per turn should be calculated in addition to (or instead of) completion percentage.
Reasoning
This stat is extremely easy to calculate from existing stats, so much so that I and other elite players and coaches
calculate it on an ad-hoc basis to find a better estimation of how offensively effective given players are than is
readily available on the AUDL stats website. It
contains much more informative information than completion percentage, and thus is a much better proxy for offensive
efficiency. Extreme examples: who cares if someone has a 99% completion rate if they only throw it backwards? I'd
rather take someone with a 98% completion rate who only throws seventy-yard goals.
Impact 🥏 🥏 🥏 🥏 🥏 / 5
This should supplant completion percentage.
Conclusion
Every one of these changes (besides expected number of goals contributed) is overwhelmingly simple to implement and would make the statistics on AUDL's website more informative. In making these changes, the AUDL can continue to push forward the state of Ultimate statkeeping.