06 May 2022
For the first time, thanks to augmented reality and 5G, a novel experiment in breast cancer surgery took place between the Champalimaud Foundation, in Lisbon, and the University of Zaragoza, in Spain. The surgery was remotely supervised by a second surgeon, but everything happened as if this second surgeon were physically in the operating room beside the first, looking over his shoulder while he performed the operation.
Yesterday, May 5th, at 3p.m. (Lisbon time), surgeon Pedro Gouveia was in the operating room, at the Breast Unit of the Champalimaud Foundation, in Lisbon, ready to start, as in so many other occasions, performing breast cancer surgery. Meanwhile, another surgeon from the same unit, the young Spaniard Rogelio Andrés-Luna, was attending the operation, and intervening, when needed, by supplying Pedro Gouveia with additional information to help him – and even guide his gestures. Everything seems to be business as usual – but it isn’t.
For starters, Pedro Gouveia was wearing mixed reality goggles (Hololens 2, developed by Microsoft). They allow him to see, by transparency, the real world around him – namely the patient lying on the operating table – and, at the same time, to have access to information projected on the goggles’ special lenses, just as if he were a fighter pilot wearing his special helmet.
It must be said that Pedro Gouveia is a pioneer in the use of Hololens goggles to perform breast cancer surgeries guided by digital, personalised models of the tumours and the patient’s body. He developed, also at the Champalimaud Foundation, a digital, non-invasive methodology that allows surgeons to very precisely locate the tumour to be extracted, almost as if the breast tissue around it had become transparent.
Coming back to the experiment that took place this thursday, it is important to highlight that Rogelio Andrés-Luna was not physically present in the operating room during the surgery – he wasn’t even in Lisbon or even Portugal. He was actually more than 900 km from the operation site, on the stage of the Congress of the Spanish Association of Breast Sugeons (AECIMA) – which takes place from May 5th to 7th at the Faculty of Medicine of the University of Zaragoza – demonstrating, jointly with Pedro Gouveia, a novel form of remote surgical supervision.
Rogelio Andrés-Luna had, as his sole work tool, a laptop computer that was linked to Pedro Gouveia’s Hololens through a private 5G network of telecommunication from providers Altice Portugal and Movistar (in Spain), and he used one of the usual Web browsers to access a dedicated software developed by remAID, a company headquartered in Germany. And, in spite of the considerable physical distance between the two doctors, everything went as if the “supervising surgeon” was right next to the “performing surgeon”, looking over the latter’s shoulder and assisting him in his delicate task.
But how was this possible, given the temporal lag introduced in images, and even more in videos, by the transmission of data via the internet and Wifi – the most powerful broadband technology generally available today for data transmission in telecommunication networks? How did the two surgeons manage to sinchronise their delicate actions with a temporal lag (the technical term is “latency”) of several seconds between what one of them was doing and the other one was seeing? The answer is that the experiment that took place yesterday didn’t resort to 4G nor to Wifi technology; instead, it used 5G, the latest and most powerful technology for transmitting digital information. “5G pulverises latency”, reducing it to a few milliseconds, says Pedro Gouveia. That is why it was so crucial in the context of our experiment”, he adds.
During the operation, Rogelio Andrés-Luna was not only able to see (almost) exactly what Pedro Gouveia was doing right at every moment, but Pedro Gouveia, on his side, was able to receive, almost instantaneously through the Hololens, useful and timely information, in particular short videos of similar surgeries previously performed. And he was even able to visualise in a timely fashion, right in front of his eyes, superposed on the patient’s body and drawn in real time by Rogelio Andrés-Luna on his laptop, a blue line indicating the site that was chosen to make the initial incision.
“We performed the first live experiment in the world, during breast cancer surgery, using what we call ‘remote proctoring’”, Pedro Gouveia enthusiastically explains.
Remote proctoring is already being used to remotely monitor written exams. In this case, the software makes it possible to supervise students via video while they are doing the exam – and also to detect possible suspect behaviours (such as the use of cell phones). Here, the latency of the images transmitted by 4G/Wifi is not a hindrance, because the images of the students are recorded during the exam and can be replayed later.
If Pedro Gouveia believes that this remote supervision methodology has the potential to become one of the key features of the operating room of the future, it is precisely because 5G takes remote proctoring to a new level in terms of simultaneity.
In his role as a tutor, says Rogelio Andrés-Luna, “I provided indications to my ‘apprentice’”. “I was able to indicate (with my blue ‘pencil’) the sites where he needed to be extra-careful, and show him images or videos. We were in constant audiovisual contact.”
To execute this feat, the two doctors relied on the collaboration, as would be expected, of telecommunications operators on both sides of the frontier. These were Altice Portugal, which was in charge of making everything work seamlessly on the Champalimaud Foundation side (this institution was equipped, with a 5G private Altice network); and Telefónica on the Spanish side, which ensured, thanks to mobile equipment installed in the vicinity of the University of Zaragoza, the 5G coverage of that site.
The success of the experiment, which constitutes a first step – a proof of concept, according to Pedro Gouveia – could change the way of doing surgery in the future, in particular not only by allowing more realistic student training, but also by providing better support to debutante surgeons as they perform their first surgeries. The latter will be able to benefit from the help/supervision of more experienced surgeons and feel much more self-assured with what they are doing.
Currently, once their training is over, young surgeons generally make their surgical debuts without any supervision – especially when they work in remote places or countries, where they are the only qualified person to perform a given surgery. Their operations can be recorded and subsequently assessed, but during the surgical act “they are on their own, and in need of help”, says Pedro Gouveia. The new approach that was tested yesterday could minimize that problem, providing beginner surgeons with this much-needed help in real time, namely from their teachers and mentors.
On a more general level, Pedro Gouveia also wants to use his new approach to allow surgery students who are not on-site to remotely attend surgical operations just as if they were there.
And he goes even farther: “the breakthroughs that immersive technologies will bring, through remote mentoring/proctoring, open a new era: that of the use of the so-called ‘metaverse’ in post-graduate medical education”, he points out. “The metaverse is defined as Internet access via augmented, virtual, mixed, and/or extended reality through a headset, and is already considered to be the next generation mobile computing platform!”, he exclaims.
He admits, though, that his idea of the operating room of the future – that he envisions as “a multimedia space” – is just taking its first steps. “We still need to validate methodologies, perform more surgeries, identify hurdles, optimise the augmented reality goggles, and so forth”, he concludes.
Photos by Helena Costa, Altice Portugal
Doutor Pedro Gouveia & Dr. Rogélio Andrés-Luna
