With all the history made at World Championships 2019, it was easy to miss perhaps the biggest news of all: this was the first competition that included the use of artificial intelligence and 3D sensors, “robot judges” if you will, as an additional judging tool.

Although the discussion of these AI judges dates back to 2017 and they have been present at competitions for making improvements and testing the technology, they have never before been used alongside human judges, making this competition one for the history books.

The use of robot judges won’t end with this year’s World Championships. Although there are no official confirmations of future use just yet, they were deemed successful at this competition, so we are sure to see them again soon. This was a historic, groundbreaking moment in gymnastics, and one that is certainly going to affect the future of the sport.

Summary

The technology was first used at the 2018 World Championships just to do technical verification, and it successfully completed multiple other tests in April 2019 at the Tokyo All-Around World Cup as well as the Junior World Championships in June. Since those test went so well, the FIG Executive Committee allowed the technology to be used at Worlds 2019. 

The technology started with vault, since the apparatus is used in both men’s and women’s gymnastics, and now has expanded to assist with vault, pommel horse, and still rings. FIG has announced that it plans to expand the technology to the other six artistic gymnastics apparatuses, but has not announced any application yet. So far the technology is only being used to assist, as it can only judge difficulty.

At World Championships, it was used only during inquiries and blocked scores. Inquiries are when a gymnast directly challenges the judge’s score, and blocked scores are when the judges themselves have large gaps between their scores. Although it was only used as backup so far, it was the cause of a few changes to scores just at this one competition. 

The International Olympic Committee has not made any official announcements as to whether or not this technology will support human judges at the 2020 Olympic Games in Tokyo. Since the technology was utilized multiple times during one this competition, one can only imagine how it would affect other competitions, such as the Olympic Games. 

Since the technology is still new, it is not considered by most to have proved itself as fully capable to judge gymnastics, so it will be awhile before robot judges are regarded as commonplace and used in all international competitions. Although the technology is expected to eventually progress further, it would still be in just a supporting role for the time being. 

This is a groundbreaking time in gymnastics history, but AI technology has already been used to assist referees in other sports. Baseball and tennis are two sports that have already begun using similar technologies, and the use of AI in sports doesn’t seem to be slowing down anytime soon. Many people have opinions on whether the technology will be good or bad for the sport, and there are many angles to consider. 

Pros

Gymnastics fans have many reasons to be excited about these new developments. FIG shared in an official press release that it sees this technology being used for more than just judging, and one way this technology could be used is as a new fan experience. There isn’t any specific information out yet on how the 3D sensors could be used to create a new viewing experience, but one can only imagine how this could change the way we watch gymnastics from home, as well as the way we receive data about the competitions. 

“I think the introduction of the robot judges is a great idea and a sign of the times,” Brett McClure, USAG men’s high performance director, said. “As technology improves globally, it must be our responsibility to improve our sport using these technologies to prevent as much human error. They can eliminate human error, make fewer mistakes, and speed up the time of the competition.” 

It also is a possibility that this technology will be implemented in gyms across different countries and used as a coaching tool. Coaches could better train gymnasts and would potentially end up with more difficult skills and cleaner form. This technology would help gymnasts at any age or level, and even coaches at the elite level are excited about it. 

“The use of new training technology is always something coaches and federations are looking for, specifically in the tagging and archiving of individual training sessions,” McClure said. “Typically there is only one to two coaches per five-plus athletes, and this technology would be able to capture, label, archive per athlete on its own for video review at a later time for the athlete and/or coach.”

This technology could also keep gymnasts from getting injured. The high injury rate has always been a critique of the sport as a whole, and this could be a turning point that keeps gymnasts safer. If coaches could see form breaks and posture issues on different skills and landings, they could change the posture immediately, before the gymnast repeated the skill many times and caused an injury.

Of course, fixing unconscious bias and human error in judging is the main reason this tool will become important. Gymnastics is often criticized for not being standardized enough. Judges do everything they can through extensive training to be fair, but unconscious bias often comes through. This occurs in every type of gymnastics, whether that be club, elite, or NCAA. 

NCAA gymnastics has especially been called out on its problematic judging, this past year even getting an email telling judges to “make sure that your scores are justifiable,” as “name recognition, team rankings, or media exposure may be affecting scores given by some judges.” 

This reputation bias frustrates gym fans and gymnastics critics alike, and this new technology could put an end to that altogether. Not only can judges have an unconscious bias, but some meets have judges working long hours that can create fatigue, and scores can deviate from that alone. Robot judges assisting would alleviate some pressure from the judges, as it can be a type of fact checker, helping judges back up their scores. 

“At large international competitions with so many athletes competing in a subdivision and multiple subdivision, it is difficult to keep the judging consistent,” Canadian gymnast and 2019 worlds competitor Cory Paterson said. “Throughout the competition, seeing skills performed in different ways and to different degrees of perfection can affect how the skills are seen and judged later on. It could be used as a reference for accuracy and to determine what might need to be fixed in the future. It can also help judges justify the executions that are given.”

Potential gymnastics fans often give up on the sport not just because of judging errors, but also because of how complex the judging system can be, making it a hard sport to understand. “If you are not a judge, athlete, or coach, it is difficult to understand how routines are judged and scores are given,” Paterson added. “If the sport becomes easier to understand, it could create the opportunity to increase popularity and the fanbase of gymnastics.”

As a retired internationally competitive gymnast and now an international judge, Chellsie Memmel can understand both sides of the issue. She shared it could be a good thing for gymnasts who feel their score was not quite accurate, saying, “You could have had a performance where robot judges could have been really helpful because that routine was fantastic and the score wasn’t quite reflected in that or vice versa.” 

Although she knows firsthand that judges try to keep scores accurate, there are times when unconscious thoughts can affect a score. “There are certain aspects of different types of gymnasts or different qualities that someone has that you are more drawn to, someone could just prefer more powerful gymnasts over artistic and vice versa, so I think there’s some of that, where that is kind of in your subconscious, where that is just your preference.”

Paterson agreed with Memmel’s opinion on unintentional judging bias. “Even if it is unintentional, being a well-known athlete or country can affect deductions given when they are borderline between .1 or .3. There’s an immense pressure, because as seen at numerous competitions, the difference between first and second can be .1 or even less.” 

Cons

Despite all the positives that could come from this new development, some people are still skeptical. Any new technology will have doubters, and some of the arguments made against the technology raise good points. 

In an interview for The Guardian, Nadia Comaneci pointed out one problem with this technology. “Gymnasts are known for pushing the skills, looking for new angles, turns, points—so what happens when someone comes along with a totally different routine that has not been seen or registered by the computer? How would that be judged?” she asked. 

So far, we don’t have the answer. But with Simone Biles debuting so many new skills lately, it is an important question. 

Skeptics have also raised the point that any technology has its problems. What would happen if we came to rely on the technology, and it crashed during an important competition? Another technology speed bump that could occur is someone hacking the technology. Fujitsu is a company with the sport’s best interests at heart, but hackers could get into the technology and make changes.

So far, the technology cannot do as much as some, including Brett McClure, hope it will be able to in the future. “They are unable to actually evaluate the execution errors properly with different body types of the athletes. I don’t see much of a help if AI only evaluates the difficulty score, which seems to have the least amount of variance from official to official,” he said. “Once the AI technology reaches a level that it can identify skills and execution errors on its own, it will be more valuable. I am encouraged on the direction they’ve gone thus far, but I think we are still in the infancy stages of this AI technology.”

Yet another critique of the technology is that it creates a further divide in the sport between gyms and countries that have money and those that do not. Using this technology at meets could be a great thing, but if some countries cannot afford to purchase it, they might get skipped over in hosting competitions. One solution would be if FIG purchases a set and uses the same robot judges for all international competitions, but we don’t yet know what the plan is. Similarly, some training gyms might be able to purchase the technology for their coaches and gymnasts to use and some might not, creating further divide between gyms and countries. 

“It could be really helpful, but with implementing it, how much is it going to be helpful and who is going to be able to afford the technology and put it in their gym?” Memmel asked. “If the information can be gathered and distributed easily enough for coaches to understand and to implement into their program, I think that could be really good, but we don’t know the potential yet.”

Role Of The Technology

While there are both possible positives and negatives this technology could bring, we will not know the effects it can have until we know what its true role will be. Many feel it can never fully take over for human judges, because although it can measure angles, height, and speed more accurately, it cannot judge artistry, choreography, or how the full routine fits together. 

We can only guess if FIG plans to implement robot judges as supportive technology or used instead of human judges, but regardless, the future of gymnastics will be fine as long as we take it slow and stay reasonable about what the technology can and cannot do. So far, FIG has given us no reason to doubt that it will fully test and carefully implement the technology, given that the tech has been around since 2017 and is just now being used in competition. 

Memmel shared the same opinion many people in the gymnastics world have voiced recently, that no matter how good the robot judges are, they cannot stand alone. “I think you’re always going to need to have human judges, because I think it’s really hard for a computer to take in actual performance quality and the artistry of what someone is doing.”

Paterson agreed with her, saying “Robot judging would only be able to use the lines and angles of the body to judge certain aspects, it wouldn’t actually see how well skills are performed. Skills in gymnastics are very complicated and there are numerous factors that occur during a routine. Because of this, there will always need to be a human judging component that understands all the rules and can adapt to the routines that athletes perform.”

One thing judges want is a clear expectation of what is expected of human judges if robot judges do become commonplace. “I think it depends just exactly what it’s going to be and how they’re going to be implemented and what the computer is going to be doing and what is expected of the person who is judging as well,” Memmel said. 

How It Works

This technology is hardly noticeable on the competition floor, as it is no larger than a typical internet router. Inside these small boxes, however, is technology that is revolutionizing the sport. The Japanese company Fujitsu partnered with FIG to create 3D laser sensors that follow the movements of gymnasts and feeds that data to an artificial intelligence system that further accesses the speeds, heights, and angles of the athletes. 

These sensors use about 2.3 million lasers per second to measure a 360-degree picture of the athlete. Basically, it uses the 3D view of the gymnast and tracks his or her movement to take the skills and turn them into numerical data, feeding that to the AI system and giving it to the judges with accurate, quick information. 

If you want to know exactly how the technology works, the company explains more here. 

Body Scans

At World Championships in Stuttgart, the gymnasts were asked to undergo a “body dimension measurement” procedure. This was a scan of their bodies, done by the AI technology, to make the judging more accurate. Because many gymnasts have different body types, some being very muscular and some more thin, Fujitsu and FIG wanted the scans to be as precise as possible to get the most accurate results. 

“I participated in the body dimension measurement and I believe all my teammates did as well,” Paterson said. “It was easy to get the measurements done, and cool to help in the development of the program for future use. If it can help further the sport of gymnastics, then why not?”

Memmel said that if she was still competing today, she would participate in the body dimension measurement procedure. “If it was coming and that’s what they’re going to use I would [do the scan], because you want it to be the most accurate for every single person who is going to compete. Every single person’s body is so different that to have as many subjects as you can would make it the most accurate for everyone.”

No one was forced to participate, and many gymnasts at worlds agreed to try it. The procedure was very quick, taking only minutes, and the information was only used to make the computing system more accurate.

McClure confirmed that the entire USAG men’s team did the procedure, but none of the women did.

We will have to wait and see how this technology is implemented in the future, but this is certainly an exciting time to be a gymnastics fan.