With the 2019 Women’s World Cup happening recently, soccer has been on my mind a lot. While watching the US team play, I noticed that several of their games were highly influenced by penalty kicks.
In the round of 16, they defeated Spain 2-1, with both goals coming from penalty kicks. The second penalty, which directly resulted in a United States victory, was considered controversial by many fans on twitter. Spanish player Virginia Torrecilla seemed to barely graze the knee of US player Rose Lavelle, an error that many thought was not strong enough to warrant such a game changing penalty attempt.
Then, in the semifinal match, the US defeated England 2-1 after a very close call where US goalkeeper Alyssa Naeher saved a poor penalty kick attempt by England. If that kick attempt had been better, England would have had a relatively high chance of tying the game.
Had the penalties in either of these games gone differently – if the controversial penalty had not been awarded vs Spain, or England had made their penalty attempt – the US team’s run in the tournament could have gone very differently.
These games, among others in the tournament, inspired an interest in studying penalty kicks. From my perspective as a viewer, penalties have always seemed a little bit too extreme, turning seemingly small penalties into game-changing scoring attempts. When the opportunity arose to study the penalty box as part of a group project at my Carnegie Mellon Sports Analytics summer research experience, I jumped on it. I wanted to see if the data supported my beliefs, and if so, whether I could work with other students to design a better solution.
An examination of the penalty area is also important and interesting because, although the sport of soccer has evolved over time, the penalty box has not changed since its addition into the game in 1902. Considering that the exact dimensions chosen for the original penalty box appear arbitrary (there is no literature explaining why the dimensions were chosen), I think it’s about time to see if we can design a better penalty box.
The Data
In our search to understand the penalty box, my group and I analyzed publicly available data from StatsBomb, available here. They provide play-by-play game data, as well as final game results and team information, for four tournaments/leagues from within the last year:
2018/19 FA Women’s Super League (England)
2018 National Women’s Soccer League (USA)
2018 Men’s World Cup (International)
2019 Women’s World Cup (International)
The idea was this: look at where penalties occurred within the penalty box, examine how likely players would have been to score if shooting a normal, non-penalty shot from that location, and compare this to their likelihood of scoring if awarded the penalty. Large differences between these two options in certain areas of the penalty box would indicate that penalties in those areas are extreme and possibly too influential on the game.
The true probability of making a shot can be challenging to quantify, but StatsBomb provides a statistic called expected goals (xG) that estimates this for us.
A closer look at expected goals
The xG statistic is calculated for each shot taken toward the goal. Expected goals for a shot range from 0 to 1, with the value representing the probability of that shot resulting in a score.
For example, a shot taken from two feet in front of the goal while the goalie is out of position would have an xG value close to 0.95 (95% chance of success), while a Hail Mary shot from 15 yards away would have an xG of <0.02 (less than 2% chance of success).
xG for any single shot is found by comparing the shot to similar shots taken before and seeing what fraction of those were successful. Similarity is measured in a variety of ways. Some aspects that are considered are:
Distance from the goal
The width/area of the goal that is not protected by the goalkeeper
Whether the ball was headed vs kicked
If assisted, whether it was a cross, through ball, or neither
Whether the shot resulted from a corner, free kick, etc.
Whether there was a breakaway / fast break leading to the shot
If we look at the expected goal values for all of the shots in the StatsBomb data, we can see that the majority of shots have a relatively low probability of success. Almost all shots have a < 25% chance of success, and many fall below 10%.
However, penalty kicks are outliers. They are successful around 75% of the time, a success rate only beat by a handful of shots. This is incredibly high for a sport as low scoring, and with such low shot success rates, as soccer (for reference, the average total number of goals scored per game for our data set is a little bit less than 3). A chance at a penalty kick gives a big advantage in a game.
While xG values certainly are certainly informative, it is also important to understand the limitations of the xG statistic. xG values are estimates. No two shots are exactly alike and xG does not account well for defender positions. As a result, sometimes the number of expected goals for a team across a tournament does not line up perfectly with what actually happens.
A look at the 2019 Women’s World Cup data confirms this. Here we see the difference between the true number of goals and the expected number of goals (calculated as the sum of xG for all shots taken) for a variety of teams in the tournament. Teams in blue scored more than expected, while teams in red scored less.
It appears that, at least for the Women's World Cup, the expected goals model seemed to underestimate the success of the teams who ended up placing well in the tournament, while overestimating the teams who finished in lower places. However, we can also see that, while xG is not perfect, it is mostly pretty accurate. The majority of teams’ expected goals were within about 1 goal of reality.
Just how much benefit do penalty kicks provide?
Once we established that expected goals were relatively reliable, we wanted to examine shot probabilities from different areas of the penalty box. Below are the maximum expected goals* for shots taken in the four different tournaments/leagues that StatsBomb has data for. The patterns look relatively similar. We see that, consistently, shots taken close to the goal and more toward the center of the penalty box appear to have the highest success rates.
Combining the shot data from all four tournaments/leagues together, we can see this pattern clearly.
Additionally, in yellow we see the locations where fouls leading to penalty kicks occurred. While the majority of penalties fell in the region directly in front of the goal, where there would have been a decent probability of shot success, quite a few penalties happened toward the outside edges of the penalty box in areas where the attacking team would have had very little chance of scoring if not awarded the penalty.
These outside areas are the extreme cases where penalty kicks provide a seemingly unfair advantage to the attacking team for what can sometimes be a very small mistake by the defense.
Examining some statistics will make the severity of penalty kicks a little bit more clear.
Penalty kicks vs normal shots
Let's start by comparing expected goals for penalty kicks to normal shots taken within the penalty box.
Normal shots from within the penalty box have the following characteristics:
Minimum xG: 0.01 (1% success)
Average xG: 0.13 (13 % success)
Maximum xG: 0.85 (85% success)
As a reminder, penalty kicks have an xG of 0.75.
The maximum possible advantage from a penalty kick, based on our supplied data (the difference between the lowest recorded xG in the penalty box and the 0.75 xG of a penalty kick) is a 7455.9 % increase in xG.
The maximum possible disadvantage from a penalty kick is a -11.6 % decrease in xG. This would only happen in the case that the attacking team is fouled very close to the goal.
The average advantage is a 462.8 % increase in xG.
For a sport where the total scores of both teams combined are, on average, less than three, these increases in xG seem a bit extreme.
Penalty kicks vs free kicks
We can also compare penalty kicks to another common soccer penalty, free kicks. A free kick is a penalty given for the same offenses as a penalty kick that is awarded if the foul happens outside the penalty box. In a free kick, rather than shooting from directly in front of the goal each time, the team awarded the penalty shoots from where the foul happened.
The locations where free kicks were taken are shown in red below, while the locations of penalty kick fouls are shown in yellow.
Free kicks have the following characteristics:
Minimum xG: 0.01 (1% success)
Average xG: 0.05 (5% success)
Maximum xG: 0.17 (17% success)
The maximum possible advantage of a penalty kick, vs a free kick, is an 8007 % increase in xG.
The minimum advantage is a 353 % increase in xG.
The average advantage is a 1269 % increase in xG.
While this comparison between free kicks and penalty kicks does not account for the fact that free kicks occur further from the goal (kicked from outside of the penalty box), we can see clearly that a penalty kick is a much more severe punishment than a free kick. The advantage of a penalty kick seems even more extreme when you consider that the only difference between the two is whether the foul happened within the penalty area. A <1 yard difference in the location where a foul happens can have a huge impact on the chance of scoring.
One solution to address this issue of penalty kicks seeming too powerful is to shrink the size of the penalty box, removing the areas where penalty kicks give extreme advantages.
Redesigning the penalty box
The technique we will use to redesign the penalty box is to restrict the penalty area to only include locations where the expected goals are above a certain cutoff.
Here we can see what our region might look like at for a range of xG cutoffs between 5% and 60%. For each cutoff, just imagine drawing a box around the highlighted blue hexagons.
The problem is choosing which cutoff is optimal. This is very subjective, and depends entirely on how much you want to penalize the defending team.
One way to justify how significant our penalty kick advantage should be is to examine fouls in other popular sports (particularly those where fans tend to view the fouls as fair) and see how large the increases in expected points/goals are.
First let’s examine basketball.
3 point fouls are more extreme than 2 point fouls, so we will look at those since we want our penalty kicks to provide a somewhat decent advantage to the attacking team.
In the 2018-19 NBA season:
3 pointers averaged 1.065 points per attempt
Free throws led to an average of 2.298 points per three attempts (3 free throws are awarded per 3-point foul)
This is a 116% increase, on average, in expected points from a 3 point foul. If we want to match this foul bonus, we would use an xG cutoff of around 35%, and our penalty box might look as follows:
(Note that the true shape of our region falling above the cutoff is a little tilted. The reason that the penalty box is drawn vertically, and not tilted at all, is because we wanted to maintain the symmetrical shape that is customary for sports fields/arenas.)
One downside to using basketball as a guide for redesigning soccer’s penalty box is that basketball is a much higher scoring sport and penalties are therefore much less influential. Three point fouls only lead to around a 1.2 point advantage in a sport where teams can often score above 100 points per game.
It might make more sense to look into a sport that has more similar final scores to soccer to see if we get different results.
We’ll examine hockey next.
In the 2018-19 NHL season, games averaged total scores of about 6 goals. This is a lot closer to soccer’s approximately 3 goals per game.
For our penalty comparison, we’ll look at power plays. A power play happens after a foul, and means that one to two players from the fouled team must leave the ice for some period of time (either 2 or 5 minutes, depending on the severity of the penalty. This gives the opposing team a one or two man advantage on the ice, and is therefore associated with an increase in scoring chances.
Looking at data from this last NHL season, we can see the strength of this advantage. In normal play, teams averaged 9.5% shot success. During power plays, they averaged 19.9% success. This is a 109% increase in expected goals. If we want to match that in soccer, we would use a 36% xG cutoff, and our penalty box design might look as follows:
Even though hockey and basketball are very different in terms of points scored, we see that the penalties we focused on from each sport result in identical looking penalty box designs. The cutoffs used for xG are different by 1% (power plays are a slightly more severe penalty than 3 point fouls), but this is not significant in leading to a change in our penalty box design.
The fact that both of these penalties in two very different sports result in similar penalty advantage is promising, and good evidence that we might have hit upon a design that would make soccer consistent with several other sports. However, before we decide on a final design, we want to consider goalkeepers.
The penalty box is not just the area where fouls result in penalty kicks. It is also the area where goalies are allowed to use their hands. Below, we can see the locations where goalies were recorded interacting with the ball. It is clear from this that the entire penalty box region is used. Simply shrinking the penalty box would limit goalies’ ability to defend the goal properly. Since we are trying to assist the defense through this redesign process, this is something we do not want.
The Final Design
In order to account for this goalie problem, we propose the following solution:
Maintain the old penalty box as the area where goalkeepers can use their hands
Draw a new, smaller box, based on our earlier cutoff analysis, that is just for penalty kicks
This new proposed penalty box redesign would likely affect future games by reducing both the frequency of and advantage from penalty kicks, but how might it have affected past games?
We can look at the locations of fouls that led to penalty kicks and the effect that those penalties had on the game result for the team awarded the kick, shown below. Fouls located in the the red-shaded regions would be affected our penalty box change.
The fouls marked in red led to a tie for the kicking team. Though it is difficult to predict the exact effect on the game results, had those not occurred, those teams would have likely lost their games.
Fouls marked in green led to a win (by 1) for the kicking team. Had those not occurred, the kicking teams would have most likely ended up tying, which in a tournament would have led to extra time / a possible penalty shootout.
Fouls marked in blue had no effect on the games where they occurred. Either the shot was missed, the kicking team scored but still lost, or the kicking team scored but would have still won without that point. Had those not occurred, we would not expect any notable changes in game results.
As we can see, there are nine circled fouls where game results might have been affected with our redesign. But what exactly would have happened with those fouls if the penalty box had not been there? They would still need to be penalized in some way, just not with a penalty kick. The first thing that comes to mind is that fouls in the shaded regions would become free kicks instead of penalty kicks. However, there is an issue with this idea. Free kicks require the defending team to be at least 10 yards away from where the kick occurs, but this would be impossible for some free kick opportunities in the shaded region due to the close proximity of our new penalty box lines to the goal.
One alternate solution I thought of would be to institute a penalty that is a blend between a free kick and penalty kick for those regions. Essentially, teams would shoot from the location of the foul (rather than the penalty spot) with only the goalkeeper defending. In that situation, fouls in locations with low xG would result in lower probability shot attempts, while fouls in locations of high xG would likely result in higher probability shot attempts. This combo free/penalty kick would keep the strength of the foul somewhat consistent, and could provide an interesting twist to traditional soccer.
As we can see, redesigning the penalty box would be a big change for the sport of soccer, but I think it’s a change that is long overdue.**
~
Bonus: Alternate designs for future consideration
Shrinking the penalty box is only one option for redesign. There are several other things we could do which we did not have time to investigate during this project that could be interesting to look at in the future. I'll describe them briefly.
1. We could move the penalty spot back. This would reduce the xG for penalty kicks.
2. We could move the penalty spot back AND make the penalty area smaller. This would reduce xG for penalty kicks as well as increase the minimum xG for shots in our penalty box region.
3. Finally, we could draw multiple penalty boxes where fouled teams can shoot from different locations depending on how close to the goal they were fouled (fouls inside the yellow region and outside the red would shoot from the yellow dot). While this design has the potential to lead to very fair penalty kicks, it would likely be difficult for referees to manage.
All of these, along with our previous design, are reasonable options for new penalty box designs, though it is difficult to know which would be best without some real-life testing.
*A note on why we used maximum xG instead of average:
Some of you might be wondering why we looked at maximum xG. Wouldn’t average be a better representation? In some ways, it is, capturing a better idea of how much advantage we would get on average from a penalty kick from different regions, rather than just the least extreme advantages we’d get from penalty kicks. However, the problem with using average xG is that, when we compare soccer to the fouls in other sports and try to draw cutoffs that would give penalty kicks a similar advantage, using average xG results in cutoff regions so small that they are within the region that ONLY the goalie is allowed to be. We didn’t want to eliminate penalty kicks completely, and maximum xG allowed us to maintain a large enough region that penalty kicks could still happen somewhat often.
**Project disclaimer:
While I argue that changing the penalty box would be a welcome change to the sport of soccer, it's important to note that it's not as simple as that. Such a huge change to a big part of the game would likely have significant affects on other areas of gameplay, some of which could even be difficult to predict. A small penalty box could certainly change the amount of pressure, as well as play styles, in the region near the goal and could lead to players clumping in the new penalty area attempting to draw fouls. It might have other effects as well. Our proposed penalty box design is just one small, preliminary piece of the puzzle. In order to truly arrive at the "best" redesign option, we would have to consider all of the impacts that a variety of designs (including some of the future work ones I mentioned) would have on different aspects of the game, and choose the design that would be most "fair" without driving away traditionalist fans by making too extreme of a change to the sport. This is certainly no easy task, but if done properly, it could have a positive impact on the future of soccer.
Special thanks to Kostas Pelechrinis (University of Pittsburgh) for leading/sponsoring this project.