Computers have already been trained how to write up quick summaries of sporting events after studying the box score, putting the jobs of sports journalists at risk. Now comes the next step: live commentary of professional sporting events as they actually happen, thanks to a combination of machine learning and computer vision.
In India, for example, computers are able to provide text-based commentary of cricket matches with 90 percent accuracy. In a recently published paper (“Fine-Grain Annotation of Cricket Videos”), a group of three Indian researchers – Rahul Anand Sharma and C.V. Jawahar, scientists at IIIT Hyderabad, together with Pramod Sankar K, of Xerox Research Center India — showed that weakly-supervised computers could reliably distinguish what’s happening during videos of cricket matches and then provide text-based commentary.
To make that possible, computers analyzed hundreds of hours of cricket videos from the YouTube channel for the Indian Premier League, breaking them into categories based on text descriptions of them that were already available. The next step was breaking down these longer videos into smaller scenes in order to classify each video shot. Then a computer algorithm had to find the right commentary that matched up with what was being shown in each video shot. For that, the researchers used commentary for about 300 matches that already existed in the Cricinfo database.
As a result, computer algorithms were able to accurately label a batsman’s cricketing shot by using visual-recognition techniques for an action that sometimes lasted no more than 1.2 seconds. “To learn such a representation, several examples are needed. A computer program then learns from these examples using machine learning algorithms, and tags parts of the video with these labels,” C.V. Jawahar told India’s NDTV.
For now, NFL and NBA fans in the United States are in the clear – there’s no immediate risk that a machine will be able to make sense of the manic action of 5-on-5 or 11-on-11 sporting events. A computer would struggle to make sense of a botched Michigan punt returned to the end zone in the final seconds by Michigan State or the bizarre sequence of events in the final seconds that led to Miami defeating Duke. And forget about it if athletes are wearing Color Rush uniforms – a machine that’s been trained to recognize certain shapes and colors might just throw in the towel before the start of the game.
However, there is one sport that might be open to real-time computer analysis sooner than you think — and that’s tennis. After all, a tennis ball is fairly easy to track on a court and computers that are able to determine exactly where a ball bounces on the court are a routine part of any live tennis broadcast. Plus, there are only two players, so there’s less action to monitor. Other than a few blazing 100 mph serves and a few furious serve-and-volleys, what’s so hard, really, about calling a tennis match between two baseline duelers?
The same Indian researchers, for example, have analyzed how a machine might be able to break down the action of a tennis match. In a recent paper presented earlier to British artificial intelligence researchers, C.V. Jawahar and Mohak Sukhwani showed how machine learning algorithms could be used to provide text commentary on tennis matches.
Of course, we’re still a long way from a live sports broadcast completely narrated by a machine, but you can see the direction we’re headed. The Indian researchers see their technique as a way to help out reporters and journalists. And, indeed, the Associated Press recently signed up to experiment with computers providing written description of college sporting events.
As machines become ever faster and ever more adept, however, it’s possible to think in terms of what machine learning means for the “play-by-play guy” and the “color commentator” — AI probably gets the play-by-play guy first, the color analyst second. After all, it’s harder to replicate so much human knowledge on the fly — a machine may be able to analyze whether Roger Federer just hit a forehand or a backhand, but may not know the full back story of how he prepared for the match or anything about his current physical condition.
However, there’s one advantage that machines have that humans don’t — and that’s the ability to process views of the action from various angles at once. How many times have you watched a sports broadcast in which the broadcasters had to watch — and re-watch- a play from many different angles before coming up with a definitive assessment about what just happened? Machines able to pull in multiple video feeds from multiple cameras at once might not have that problem.
For now, the play-by-play ability of machines is not expected to take away any jobs. The focus is on using this AI technology for training and coaching. According to the researchers, the annotation of the videos allows computers to search across hundreds of hours of content for specific actions that last only a few seconds.
As a result, imagine preparing for a tennis match with a rival and having an AI companion that can immediately make sense of your opponent’s playing style and search for a specific stroke on a specific part of the court – within seconds. Watching hours of tape is already a staple in the professional sports world – but now it appears that machines are becoming just as proficient as humans at making sense of all the nuances hidden in that tape.
For now, AI does not pose a risk to veterans of the sports broadcast booth. For reporters and journalists on the sidelines, though, it may be a completely different story – by the time they’ve cranked open their laptops, their machine rivals may have already written and posted their article online.