Augmented reality is the more modern term for mediated reality where our perception of the world is modified in some way. The simplest of these devices could be a rear-view mirror on a car; however, cutting edge augmented reality as we think of it today began in 1958 with Heads-up displays (HUD) for fighter pilots [9]. This overlay of computer-generated imagery has progressed since that time and has become infused into our vehicles, our smart phones and now our eyeglasses.

As some authors have suggested, a HMD may cause information overload [10]. The user interface (UI) becomes important in order to tailor the content provided by the device. Google Glass uses a touchpad located on the side of the glass and voice commands in order to interact with the content of glass. Most of the newer devices have Bluetooth® (Bluetooth SIG, Kirkland, WA) connectivity and will allow for interface with a smartphone and computer, which can be used as the UI. The Technology Partnership (Melbourne, UK) has integrated electrodes capable of interpreting extraocular muscle activity as a proxy for eye tracking, which is implemented as the UI. Olympus Optical (Shinjuku, Japan) is one of the many users that have experimented with hand gesture recognition as the UI.

Google recently received recognition for their consumer smart glass (Google Glass) that did not require linkage to a device in order to display information. Google Glass is a computer with an optical head mounted display comparable to a 25″ HD television viewed from 8 ft. It uses bone conduction for audio, stores 16GB, possesses a 5MP camera, Wi-Fi, Bluetooth, voice recognition and has roughly 4 h of continuous, active battery life. It is incorporated into a familiar eyeglass form factor (Fig. 6.4).

Inventors continue to push the envelope. Babak Parviz, an affiliate professor of electrical engineering at the University of Washington, has already created a contact lens with a single LED, and it is powered wirelessly with radio-frequency (RF) energy (Fig. 6.5) [11]. This is just the beginning. More recent works have shown the creation of contact lenses capable of sensing tear glucose concentrations [12].

Within our own group, the smart phone changed the way we performed telemedicine [6]. With the advent of Google Glass, we were able to further integrate telemedicine into our practice.

As a surgical limb salvage group, we consulted with another surgical colleague and discussed requirements for resection and subsequent admixture of an antimicrobial bioactive implant to deliver into a previously infected bony defect following a combined vascular-soft tissue reconstructive limb salvage procedure (Fig. 6.6a–d). The Google Hangouts™ application was managed entirely by the operating surgeon using hands-free voice control. Additionally, real time diagrams and MRI measurements were developed as a picture in picture, permitting two colleagues to effectively consult without the surgeon’s eyes leaving the operative field to view intraoperative imaging along with surgical anatomy.

A consultation for the above patient was continued in clinic the next day. The clinician performed a dressing change along with the virtual consultant. The consultant was able to be virtually present for the first postoperative dressing change in the clinic, satisfactorily managing postoperative care.

In a different patient at high risk for wound complications, we utilized Google Glass as an educational adjunct. A junior resident donned Google Glass during a scheduled delayed primary closure of a plantar defect and was engaged in an interactive “screen share” feature, which fed detailed descriptions on retention suture technique. This allowed for real-time, visual instruction in collaboration with a senior attending surgeon (Fig. 6.7a, b). This approach maximized hands on experience and autonomy for the resident. This was performed using bandwidth from a standard 3G CDMA connection using Glass-Bluetooth-Phone. A similar procedure followed demonstrating compartmental anatomy and assisting in planning a surgical decompression of a limb-threatening infection (Fig. 6.8a, b). Similarly, investigators have shown that AR education can shorten the learning curve during the acquisition of laparoscopic skills [13].

The above example is merely one of many ways that physicians are beginning to scratch the surface of what can be made possible.

Surgeons are constantly faced with the task of mentally integrating two-dimensional radiographs and the three-dimensional surgical field, which is the very reason that augmented reality is so attractive. Researchers have been attempting to overcome this complexity for the last two decades [14].