The RETINA project stands at the forefront of photonic technology, driving innovation with its pioneering solutions across crucial sectors of healthcare, automotive, and agriculture. By harnessing the power of advanced photonic-based sensory systems, RETINA is set to redefine operational environments, showcasing unmatched reliability, commercial viability, and potential for widespread replication.
-
Healthcare
Our photonic sensory systems offer revolutionary capabilities in surgical imaging. By identifying tumorous cells and monitoring blood perfusion, we provide surgeons with invaluable real-time data, enhancing surgical precision and patient outcomes.
-
Automotive
In the automotive sector, RETINA’s advanced LIDAR and AI-driven technologies are integrated into ADAS for superior collision detection systems. This technology promises to make autonomous driving safer and more reliable.
-
Agriculture
For agriculture, RETINA’s precision viticulture solutions leverage hyperspectral imaging to manage hydric status and predict pathogen infections. This leads to smarter, more sustainable agricultural practices.
Healthcare
Automotive
Agriculture
The RETINA project stands at the forefront of photonic technology, driving innovation with its pioneering solutions across crucial sectors of healthcare, automotive, and agriculture. By harnessing the power of advanced photonic-based sensory systems, RETINA is set to redefine operational environments, showcasing unmatched reliability, commercial viability, and potential for widespread replication.
Healthcare
Our photonic sensory systems offer revolutionary capabilities in surgical maging. By identifying tumorous cells and monitoring blood perfusion, we provide surgeons with invaluable real-time data, enhancing surgical precision and patient outcomes.
Read MoreAutomotive
In the automotive sector, RETINA's advanced LIDAR and AI-driven technologies are integrated into ADAS for superior collision detection systems. This technology promises to make autonomous driving safer and more reliable.
Read moreAgriculture
For agriculture, RETINA's precision viticulture solutions leverage hyperspectral imaging to manage hydric status and predict pathogen infections. This leads to smarter, more sustainable agricultural practices.
Read moreRevolutionizing Healthcare with Multimodal Surgical Support
RETINA’s advanced multimodal sensing system is set to transform surgical procedures by providing surgeons with real-time insights into tissue characteristics.
Enhanced Tissue Visualization
The system precisely detects tumor cells and identifies critical anatomical structures, providing surgeons with a clearer visual map during complex procedures. This can lead to more complete tumor resections and preservation of healthy tissue.
Perfusion and Oxygenation Assessment
Beyond structural insights, the system assesses blood perfusion and oxygenation levels in tissues, offering critical information about tissue viability. This is invaluable for guiding surgical decisions, particularly in vascular and reconstructive surgery.
Automotive
The automotive industry is rapidly advancing towards fully autonomous vehicles, but ensuring safety in all conditions remains a critical challenge. Our multimodal sensing system addresses this by providing comprehensive obstacle and collision detection, even under complex and adverse weathering conditions.
All-Weather Obstacle Detection
For safe collision avoidance, autonomous vehicles must accurately detect and measure the position, size, and shape of people and objects. Our system excels at this, ensuring reliable performance regardless of challenging weather or light conditions.
Distance and Motion Analysis
Beyond static object detection, the system precisely measures distance and relative motion of objects ahead of and around the vehicle, providing crucial data for predicting trajectories and preventing collisions.
Integrated Sensor Array
This capability is powered by a strategic combination of sensors: our Long-range LIDAR offers high-precision 3D mapping and ranging, while a Commercial Camera provides essential visual context, enhancing the system’s ability to interpret complex scenes.
This holistic approach to sensing allows autonomous vehicles to perceive their environment with unprecedented clarity and reliability, significantly improving safety in diverse driving scenarios.
While commercial FMCW LIDAR systems for Advanced Driver-Assistance Systems (ADAS) have made significant strides, our approach builds upon these foundations to deliver enhanced performance and integration.
Field of View Coverage
Typically achieved through 2D mechanical scanning, which can introduce complexities in form factor and durability.
System Configuration
Often involves separate laser and detector components, leading to larger, less integrated systems.
Key Specifications (based on commercial announcements)
Typical range on 10% target: 150-200 m
Typical FOV: 120° x 30°
Typical angular resolution: 0.05 – 0.2 degree (often high resolution in a Region of Interest (ROI), not full FOV)
what are the improvements of the pic-based LiDAR?
Our PIC-based LIDAR system, with its full chip integration and advanced beam steering, addresses many of these limitations, offering a more compact, robust, and cost-effective solution for future ADAS implementations. This integration also promises enhanced reliability and immunity to interference.
Photonic Integrated Circuits (PICs) are at the forefront of LiDAR technology, offering a pathway to smaller, more efficient, and robust sensors essential for advanced driver-assistance systems (ADAS) and future autonomous vehicles. The RETINA project has made significant strides in developing these next-generation LiDAR systems through iterative design and manufacturing runs.
The 1st LiDAR PIC Run: Laying the Foundation
The initial LiDAR PIC run was a crucial first step, validating the fundamental concepts and demonstrating the feasibility of integrating photonic components onto a single chip. While promising, this phase identified areas for optimization and refinement to achieve the performance and compactness required for automotive applications.
The 2nd LiDAR PIC Run: Significant Advancements
Building upon the lessons from the first run, the second LiDAR PIC run incorporated several key improvements that push the boundaries of integrated LiDAR technology.
These advancements are critical for widespread adoption in the automotive industry, addressing previous limitations and enhancing overall system performance and reliability.
Key Improvements in the 2nd Run
- Fully-Integrated LiDAR without External Lens: This breakthrough eliminates the need for bulky external optical components, leading to a much more compact and robust sensor. It simplifies manufacturing and reduces costs, making the technology more viable for mass production.
- Reduced Number of Control Pads: Streamlining the control interface minimizes complexity, reduces potential points of failure, and allows for more efficient electrical integration within the vehicle’s electronic systems.
- Reduced Chip Size: Miniaturization is a continuous goal in photonics. A smaller chip size not only reduces material costs but also allows for easier integration into various vehicle designs, including hidden placements.
Agriculture
The RETINA project extends its innovative photonic solutions to the agricultural sector, specifically targeting precision viticulture. By providing a tailored multi-modal monitoring solution, we aim to revolutionize vineyard management, enabling growers to make data-driven decisions that enhance crop health, optimize resource usage, and forecast yields with unprecedented accuracy.
Water stress monitoring
A novel methodology for assessing vineyard water status, crucial for optimizing irrigation and ensuring grape quality. This goes beyond traditional methods, providing real-time, precise data on vine hydration.
Early Detection of Disease
In-field detection of grapevine trunk diseases (GTD), which are devastating to vineyards. Early detection means faster intervention, preventing widespread economic losses.
Productivity Forecast
Real-time estimation of harvest quantity, allowing vineyards to better plan for logistics, labor, and market supply. This predictive capability significantly improves operational efficiency.
Integrated Sensor Systems:
Ground-Based System:
Designed for close-range, highly detailed data collection, ideal for individual vine analysis.
- Short-range LiDAR: For precise 3D mapping of canopy structure and berry clusters.
- SWIR & VNIR Spectral Imagers: To capture detailed spectral signatures indicative of plant health, water content, and disease markers across a broad spectrum.
Drone-Based System:
For broad-area coverage and efficient data acquisition across large vineyards, providing a macro-level view.
- Long-range LiDAR: For mapping entire vineyard topography and canopy volume.
- SWIR & VNIR Spectral Imagers: Complementing ground-based data, offering a wider perspective on vineyard health and variability.
State of the Art vs. RETINA
The current state-of-the-art methods in viticulture management are often labor-intensive, time-consuming, and can lack the precision offered by advanced sensing technologies. RETINA’s multi-modal approach significantly enhances these capabilities, providing more accurate and timely insights.
Traditional Water Stress Monitoring:
- Indirect Methods: Rely on soil moisture sensors or atmospheric measurements, which may not accurately reflect the vine’s physiological water status.
- Direct Methods: Such as leaf water potential (pressure chamber) are highly accurate but destructive, labor-intensive, and provide only spot measurements.
- Remote Methods: Spectral indices (e.g., NDVI, NDWI) offer a broader view but may lack the specificity to differentiate subtle stress levels.
Current GTD Detection:
- Laboratory Analysis: Requires destructive sampling and lab time, delaying intervention.
- Visual Assessment: Highly dependent on expert knowledge, time-consuming, and prone to human error, often detecting disease only at advanced stages.
- Remote Sensing: While emerging, often lacks the resolution or specific spectral bands needed for early, precise detection.
Current GTD Detection:
- Manual Yield Estimation: Relies on historical data and expert judgment, which can be inaccurate due to year-to-year variations.
- Vision-Based Techniques: Often limited by lighting conditions, occlusion, and the complexity of analyzing fruit in dense canopies.
- Index-Based Techniques: Provide general estimates but may not account for spatial variability within the vineyard.
RETINA’s integrated LiDAR and spectral imaging solutions provide a significant leap forward, moving beyond these limitations to offer highly precise, non-destructive, and scalable monitoring capabilities.
Physiological and Phytosanitary Algorithms
Developing robust algorithms for precision viticulture requires extensive and high-quality data collection. The RETINA project conducted comprehensive data acquisition campaigns in real-world vineyards to train and validate our physiological and phytosanitary algorithms.
Vineyards Data Collection
- Images taken from July 1st to October 10th, 2024: This extended period ensures capture of data across various phenological stages of the vine, from veraison to harvest, allowing for comprehensive algorithm training for different physiological states.
- Locations: Douro and Dão, Portugal: Data was collected from two distinct, renowned wine regions in Portugal. This geographic diversity ensures the algorithms are robust to different soil types, climate conditions, and grape varietals.
- VNIR and SWIR Cameras + GPS: The setup includes Visible-Near Infrared (VNIR) and Short-Wave Infrared (SWIR) cameras to capture broad spectral information crucial for plant health analysis, complemented by GPS for accurate geotagging of data points.
- Processing Unit, Touch Screen, Battery: This portable and integrated system facilitates field operations, enabling real-time monitoring and immediate feedback.
- White Target Calibration: A white target is used for radiometric calibration of the images, ensuring consistent and accurate reflectance measurements across different lighting conditions.
Despite the meticulous planning, real-world data collection presents challenges. Out of approximately 700 images focused on the VNIR band (300 in Dão, 400 in Douro), only about 10% were deemed “OK” due to issues like repeated images, improper exposure, and blurring. These challenges highlight the need for robust image processing and quality control in future data acquisition efforts.
Camera Specifications
VNIR Camera
- Model: MV1-D2048x1088-HS02-96-G2-10
- Sensor: IMEC CMV2K-SM5x5-NIR CMOS
- Lens: 10 mm
SWIR Camera:
- Model: MV4-D640I-D01-HS06-GT
- Sensor: SCD-Cardinal640 SWIR
- Lens: 8 mm
Next Steps and Existing Datasets
The RETINA project continues to evolve, with clear next steps identified to further refine our photonic solutions for both viticulture and other high-impact applications, including medical use cases. Our work is supported by a foundation of existing datasets, which provide invaluable resources for training and validation.
Next Steps for Advancement:
- Estimate Relevant SWIR Bands for Viticulture: Based on ongoing planned experiments, we will precisely identify the specific Short-Wave Infrared (SWIR) bands most indicative of water stress, disease presence, and productivity in grapevines. This will optimize sensor design and data analysis for maximum relevance and efficiency.
- Estimate Relevant SWIR Bands for Quantum Dot (QD) for Medical Use Case: We will extend our spectral analysis to medical applications, focusing on Quantum Dot (QD) response. This will involve new phantom studies and utilization of high-resolution SWIR sensors to pinpoint optimal spectral ranges for specific medical diagnostics.
- Mimic QD Response from High-Resolution SWIR Snapscan Spectral Images: By simulating QD behavior using advanced spectral imaging, we can accelerate development and testing of medical diagnostic tools without the need for extensive physical prototypes.
- QD Sensor Manufacturing for RETINA Use-Cases: Our Quantum Dot sensor manufacturing capabilities will be leveraged to produce new samples precisely tuned to the relevant peak wavelengths identified for both viticulture and medical applications, ensuring optimized performance and specificity.
Article about RETINA was published in Photonics Spectra
