Ultra-thin imaging fiber endoscope using a diffuser as encoder and a deep neural network as decoder
Keywords
endoscopy, diffuser, artificial intelligence, coherent fibre bundle, CFB, 3D imaging, minimally invasive, neural network
Invention Novelty
Researchers of Technische Universität Dresden (TUD, Technical University of Dresden) propose a concept for minimally invasive real-time single‑shot 3D lensless fiber endomicroscopy. The technology comprises a coherent fiber bundle (CFB) based imaging endoscope which features a diffuser at the object-facing distal side of the CFB that provides a speckle pattern. Advanced artificial intelligence-based image processing techniques allow to retrieve the object information from the detected speckle image.
Value Proposition
Minimally invasive access to physiological structures is key for the operating room of the future. Conventional fiber-based endoscopes for 2D imaging rely on bulky imaging optics to transmit an intensity pattern through the fiber. The inventors propose to use a diffuser plate instead of a lens system, allowing the realization of much smaller diameters and 3D imaging. Additionally, with this setup the complex signal transmission properties of the CFB and/or the diffuser can be largely ignored since (1) the signal processing relies on a 2D intensity pattern (no need for phase information) and (2) potential defects are accounted for during the learning/training step of the signal processing.
Scheme and principle of a diffuser endoscope. Taken from Kuschmierz et al., Light: Advanced Manufacturing, 2021, Fig. 5a, unchanged, licensed under a Creative Commons Attribution 4.0 International License, creativecommons.org/licenses/by/4.0/.
Technology Description
By placing a commercial optical diffuser (or alternatively, a randomly structured diffractive optical element, for example realized by 3D-printing onto a glass plate) in the far field of a CFB, the object 3D information is encoded in an intensity-based 2D speckle pattern. Intensity patterns can advantageously be relayed through a CFB to its proximal end, where the transmitted light is ultimately detected by a camera. Pretrained neural networks for object reconstruction can then be used to immediately retrieve 3D properties from the speckle pattern of an imaged object.
Commercial Opportunity
We are looking for a licensing or/ and cooperation partner.
Development Status
A prototype of an ultra-thin diffuser-based imaging system has been realized at TUD. A proof-of-concept study shows that a trained deep neural network can reconstruct 2D objects or sparse 3D objects at a random and unknown distance from speckle patterns.
Patent Situation
German patent DE102021102755 has been granted, a US patent application has been filed.
Further Reading
Kuschmierz et al. Light: Advanced Manufacturing (2021)2:30; doi.org/10.37188/lam.2021.030
Kuschmierz et al., Proc. SPIE PC12137 (2022); doi.org/10.1117/12.2624590 (
conference abstract)
Institute of Origin
