Andrew Kadis, Peter J. Christopher, Youchao Wang

Published July 24, 2019 © MIT

Low-Cost Spatial Light Modulator Interface Board

A low-cost SLM driver board would drastically increase the proliferation of advanced microscopy and holographic displays.

IntermediateWork in progressOver 4 days2,413

Low-Cost Spatial Light Modulator Interface Board

Things used in this project

Hardware components

Intel Cyclone V FPGA

Jumper wires (generic)

FadeCandy UMFT601A-B

Story

Summary

This project aims to develop a low-cost spatial light modulator universal driver board. An SLM modulates the amplitude or phase of light; this innocuous active optical element is extremely important in the study of biological systems, enabling beam shaping techniques to be applied in super-resolution microscopy imaging for biological structures. Additionally, an SLM can be used in an inverse configuration and combined with holographic techniques to produce a 3D holographic display to faithfully reproduce the 3D structure of complex biological systems.

Currently, the electronics to drive an SLM is prohibitively expensive (order of 10 to 100 thousand USD per unit), restricting their use to niche industrial and scientific applications. This project sees to develop a low-cost (under 200 USD) SLM driver board. This would be transformative in making high-end microscopes and the research they enable more accessible to industry and academia.

The Problem

Spatial light modulators (SLMs) are liquid crystal (LC) able to control both the amplitude and phase of light. Typically constructed from large arrays of pixels each controlled independently, they can be thought of a more exotic cousin of the LC displays found in TVs and smartphones. As they capable of affecting phase, SLMs can be combined with a coherent light source to enables advanced applications such as advanced microscopy and holographic visualisation.

The University of Cambridge’s Centre of Molecular Materials for Photonics and Electronics (CMMPE) has a long history of developing new liquid crystal materials that can be incorporated into bespoke SLMs. However, one of the key limitations is the electronics required to drive them. The electronics to drive an SLM have extremely strict requirements; they need high voltages to actuate their crystals, timing requirements are stringent and generation waveforms must be very accurate or the crystal breaks down. Standard laboratory equipment can suffice for smaller liquid crystal arrays (less than ~10 cells), but these techniques do not scale up to HD and 4K resolution devices. To exacerbate this issue, many of the more interesting applications for SLMs require the image to be displayed very fast (for context, the system proposed below is targeting 2400 FPS (frames per second). What is truly needed is a low-cost SLM interface board; the CMMPE has thousands of low-cost ferroelectric SLMs but no means to drive them.

A low-cost ferroelectric SLM

SLMs are typically sold with the interface electronics designed for that specific SLM. These units are prohibitively expensive (units can routinely cost 100 Thousand USD), limiting their usage to specific high value niches in the industrial and research fields. Fundamentally though the electronics are not too dissimilar from what is found in a modern LCD display on a TV, monitor or smartphone; the development of a low-cost driver board shall provide a platform for researchers to investigate the use of SLMs for novel imaging, sensing and visualisation purposes.

Application

The key capability of SLMs is their ability to control the phase of light. SLMs form an intrinsic component in ‘state-of-the-art’ microscopy (1). This suite of tools is an extremely powerful set of techniques for studying and understanding complex biological systems.

Example of super-resolution microscopty; here we have a spatial light interference microscope (SLIM) which incorprates a SLM

Super-Resolution Microscopy: SLMs are used as adaptive optics for correcting aberrations in state-of-the-art super-resolution microscopy such as single-molecule switching (SMS) simulated emission depletion (STED) and structured illumination (SIM) microscopy (2) as well as phase contrast imaging.
Holographic Displays: By combining an SLM with holographic techniques, it is possible to build a fully immersive Holographic Display. This display is capable of presenting a full 3D structure within a large volume, with content natively adapting to viewpoint and depth-of-focus of the eye (4). This level of fidelity in displaying a 3D object is not feasible with auto-stereoscopic displays or other contemporary techniques.

SLMs find extensive use within biological systems; these include cellular imaging, neuroscience, fluorescencemicroscopy, nanoscopy, medicine, robotic surgery visualisation and more. It is envisioned that a low-cost, high-speed SLM driver board would act as a powerful driver for research and proliferation of these fields.

Outcomes and Benefits

A low-cost SLM driver board could have a potential transformative effect, opening up SLMs to a host of applications which current devices are too expensive for. An appropriate analogy would be low-cost LCDs: initially these drove applications such as pocket calculator and laptops and have recently led to the of gadgets such as smartphones and smartwatches. A similar disruptive effect is envisioned for low-cost SLMs. They are currently boutique electro-optical devices used in high-end systems, but by bringing the costs down they could make advanced microscopy, imaging and display technologies commonplace and lead to future applications.

System Architecture

The SLM being developed for is the DisplayTech HDP-1280-2. By leveraging these, it is possible to build a low-cost system with a high resolution capable of updating at high speeds. The key challenge of this project is to develop a system capable of providing the high data bandwidth required to achieve a throughput of 2400 FPS.

System Architecture

Key features of this architecture are:

FPGA: The HDP-1280-2 uses a custom high-speed interface to implement a 2400 FPS Update Rate. A FPGA is a piece of programmable hardware which is well suited to high-speed digital communications.
USB 3.0: USB 3.0 has been selected as it provides a high bandwidth data interface between a PC and the FPGA. The selected USB Controller automatically converts a high-speed serial interface into a 32-bit wide FIFO parallel interface well suited for the HDP-1280-2.

Development Breadboard

By utilising a COTS (commercially available off-the-shelf) development boards, the team has been able to rapidly construct a platform to develop the FPGA firmware and PC interfacing software. This allowed the FPGA firmware to be developed in parallel with the interface board.

Note that a custom breakout board was constructed in order to breakout the small connections from the SLM to the development board.

Development breadboard system. SLM is on the far left.

SLM Interface Board Rev1

An initial first-spin of a custom high-speed board has been developed to eventually enable 'plug-and-play' of the SLM with a PC to drive it.

Custom, bespoke high-speed SLM interfacing board.

SLM Interface Board Rev2

There were several issues with the Rev1 Interface Board. Althouhg minor, they prevented the FPGA from electrically communicating with the SLM. These were rectified in an additional spin of the board, resulting in the SLM Interface Board Rev2.

2nd Revision of the SLM Interface Board. It looks very similar, but this one actually works.

The 2nd revision of the hardware provided approriate electrical performance and signal integrity to support the HDO-1280-2. It is the board which was used to output the test images shown below.

FPGA Firmware

This has proven to be the most challenging part of the project. Due to the complex nature of the interface and the various sequences required, development of the FPGA firmware is still ongoing. Nevertheless it is possible to output a number of distinct test patterns as show below.

Successful communications between the FPGA and the SLM; above shows successful SPI communications on a logic analyser

PC Interfacing Software

A custom, C# Application has been built to drive the SLM from a PC. It has been extended as required during testing; The current version is able to support 9 distinct test images. The plan is that eventually, the user shall be able to drop custom images into here and they shall be displayed on the SLM.

Skeleton of a C# application to develop the FPGA firmware and eventually drive the SLM from a PC

Outcomes

The system is able to display 9 distinct Binary-Phase Test Images on the HDP-1280-2 SLM. They are shown below. Note the following:

The images are binary-phase and hence a piece of polariser has been placed above the SLM to see the Test Image effect.
The SLM pixels are very fine (5.6 microns). Hence it is extremely difficult to see individual pixels or even groupings of 8 pixels. Hence the features in the images below have been set to be 32 pixels wide to enhance visiblity. In practice this can be refined depending on the optical requirements of a given system.

The System is able to reliably display a number of discrete test images on low-cost hardware

Development Progress

The key outcome of this project is a system capable of displaying 9 different test images on the HDP-1280-2 SLM. In order to achieve this, the following steps were involved in practice:

A system architecture was developed capable of interfacing with the HDP-1280-2 SLM
An off-the-shelf Intel FPGA development board was used to interface with the HDP-1280-2 using a custom breakout board
FPGA Verilog code was developed to interface with the HDP-1280-2 custom data interface
A custom low-cost SLM Interface Board was designed, manufactured and assembled
Electrical issues were encountered with the breakout board interface (incorrect voltage levels and poor grounding). Signal integrity was poor and development was migrated to the Interface Board Rev 1.
Codebase from the Intel FPGA was migrated to the Lattice FPGA on the Interface Board Rev1
Hardware Issues were found with the Interface Board Rev1; the pinout on the main flexi-connector between the PCB and the SLM was incorrect and there were some modifications to the power supplies to support the new voltage levels
A second version of the interface board, Rev 2 was designed and manufactured
Custom FPGA firmware on the Rev2 interface board was used to successfully communicate with SLM
A custom PC-Based GUI program was built to communicate with the SLM via the FPGA
System able to display 9 different test images; scope to extend to further test images

Follow On Plans

Optical Demonstrator System

Although a low-cost interface board has been developed, it is of limited value without a suitable demonstration. To this end, an optical demonstration system is currently being built.

Fusion 360 Model of proposed Optical Demonstrator System

The intention is for this image to project onto a raw camera image sensor. By reducing the region of interest (ROI) of the camera, it is possible to boost acquisition speeds past 2 kHz and this verify the functionality of the SLM at higher speeds. Hence this system shall be of development value as well as being useful for demonstration.

Improve System Capability

A limitation of the current system is that it is only working in the low-speed SPI Test Mode, and not the full high-speed data interface between the Lattice FPGA and the SLM.
Hence although the current system is updating images at 100Hz, it takes approximately 10 seconds to send a new image to the SLM. Enabling the high-speed data mode shall radically improve this update rate by orders of magnitude.
Currently the GUI is only able to send 9 discrete test images. It is desirable to extend the GUI to enable sending arbitrary images.

Integrate into Additional Systems

It is highly desirable to identify external collaborators and integrate the SLM Interface Board with their systems.
It is envisioned that individual applications shall drive custom requirements such as hardware triggering, synchronised SLM updates with external components and customised timings
This approach shall evolve the capability of the SLM Interface Board and simultaneously demonstrate its value

Summary

This project set out to develop a custom low-cost interface board for the DisplayTech HDP-1280-2 SLM. The key aim of the project has been met, it is possible to phase-modulated image on the SLM using the custom low-cost hardware. In this respect, the project has been a success.

However, it was not a clear path to get there; custom electronics were developed and major hardware issues were experienced. In all, it was only possible to display an image with 3rdpiece of hardware used in this project, the Rev2 Interface Board. Nevertheless, this is par for the course for high-end electronics development and the persistence eventually paid off.

The system is capable of displaying 9 different test images on the SLM. An optical system to use the SLM ‘in-anger’ is currently being built. However, what would be fantastic would be the opportunity to engage with research collaborators and deploy the low-cost SLM interface board in additional applications.

Working with this board offers the opportunity to implement SLMs in applications where they might otherwise be too expensive. Additionally, it opens the opportunity for a level of customisation and advanced features (such as hardware triggering, synchronised SLM updates with external components and customised timings) not able to be achieve with a typical COTS (commercially available off-the-shelf) SLM.

Opportunties for Collaboration

If you are interested in collaborating with us, then please contact us via Hackster.

References

1) C. Maurer, A. Jesacher, S.Bernet and M. Ritsch-Marte. What Spatial Light Modulators can do for Optical Microscopy

2) M. Booth, D. Andrade, D.Burke, B. Patton and M. Zurauskas. Abberations and Adaptive Optics in Super-Resolution Microscopy

3) A. Kaczorowski, S. J.Senanayake, R. Pechhacker, T. Durrant, M. Kaminski and D. F. Milne. Real-Time Holographic Solution for True 3-D Display

4) A. Linnenberger. Advanced SLMs for Microscopy

5) E.Min, M. E. Kandel, C. J. Ko, G. Popescu, W. Jung and C. Best-Popescu. Label-free, multi-scale imaging of ex-vivo mouse brain using spatial light interference microscopy.

Code

MainWindow.xaml.cs

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO.Ports;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;

namespace HoloRelay
{
    /// <summary>
    /// Interaction logic for MainWindow.xaml
    /// </summary>
    public partial class MainWindow : Window
    {
        public MainWindow()
        {
            InitializeComponent();
        }

        // Helper function to setup our Serial Port for Tx
        private SerialPort setup_serial_port()
        {
            // Open a 8N1 Serial Port which is at COM3
            SerialPort test_com_port = new SerialPort();
            test_com_port.PortName = "COM3";
            test_com_port.DataBits = 8;
            test_com_port.StopBits = StopBits.One;
            test_com_port.BaudRate = 115200;
            test_com_port.Parity = Parity.None;
            test_com_port.Open();
            // Timeout after 100ms read
            test_com_port.ReadTimeout = 100;
            return test_com_port;
        }

        // Helper function to close our Serial Port after Tx is done
        private void close_serial_port(SerialPort serial_port)
        {
            serial_port.Close();
        }

        // Helper function to read all of the data recv on serial port into our byte buffer and formats as a nice string
        private string Read_serial_data(SerialPort serial_port)
        {
            // Rx Buffer
            const int RX_BUF_SIZE = 4096;
            byte[] rx_buf = new byte[RX_BUF_SIZE];
            int num_bytes_read = 0;
            // Use a try-catch as we only want to read until timeout
            try
            {
                num_bytes_read = serial_port.Read(rx_buf, 0, rx_buf.Length);
            }
            catch (TimeoutException e)
            {
                // This is simply to catch a timeout when reading, part of normal operation so do nothing with it
                e.ToString();
            }
            // If received no bytes than simply return a null string
            if (num_bytes_read == 0)
            {
                return "";
            }
            else
            {
                // Trim rx buffer to be only what we read
                byte[] rx_bytes = new byte[num_bytes_read];
                Array.Copy(rx_buf, rx_bytes, num_bytes_read);
                // Convert to a nicely formatted string and return
                string rx_string = BitConverter.ToString(rx_bytes);
                rx_string = rx_string.Replace("-", "");
                return rx_string;
            }

        }


        // Helper function to handle tx of a buffer and returns a nice string of what was sent
        private string Send_serial_data(byte[] tx_buf, SerialPort serial_port)
        {
            // Send over Serial Port
            Int32 offset = 0;
            serial_port.Write(tx_buf, offset, tx_buf.Length);
            // We wait 100ms here 
            Int32 after_uart_wait_time_ms = 100;
            Thread.Sleep(after_uart_wait_time_ms);
            // Get nice formatting for what we just sent
            string tx_string = BitConverter.ToString(tx_buf);
            tx_string = tx_string.Replace("-", "");
            return tx_string;

        }


        private void Send_test_sequence_Click(object sender, RoutedEventArgs e)
        {

            // Open Serial Port
            SerialPort fpga_com_port = setup_serial_port();

            // Send Various Sequences in little-endian
            // This is a test Bench here, so just mix and match as you want
            // Send RESYNC
            // Short
            //byte[] RESYNC = { 0x41, 0x6F, 0xDC, 0x1E };
            // Extended
            byte[] tx_data = {
                // Reset - we transmit in little-endian
                0x1E, 0xDC, 0x6F, 0x41,
                // StartOfPacket
                0x74, 0x1B, 0x8C, 0xD7,
                // Command
                0x01, 0x00, 0x00, 0x00,
                // NumOfPayloadWords
                0x04, 0x00, 0x00, 0x00,
                // Payload
                0x10, 0x11, 0x12, 0x13,
                0x20, 0x21, 0x22, 0x23,
                0x30, 0x31, 0x32, 0x33,
                0x40, 0x41, 0x42, 0x43
            };
            //byte[] RESYNC = { 0xFF, 0xFF, 0xFF, 0xFF, 0xD7, 0x8C, 0x1B, 0x74, 0x00, 0x01, 0x02, 0x03 };
            // Long
            //byte[] RESYNC = new byte[128];// { 0x41, 0x6F, 0xDC, 0x1E, 0xD7, 0x8C, 0x1B, 0x74 };
            //for (byte i = 0; i < RESYNC.Length; i++)
            //{
            //    RESYNC[i] = i;
            //}
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }

        private void Send_resync_Click(object sender, RoutedEventArgs e)
        {

            // Open Serial Port
            SerialPort fpga_com_port = setup_serial_port();

            // Send RESYNC
            byte[] tx_data = { 0x41, 0x6F, 0xDC, 0x1E };
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }

        private void Send_start_of_packet_Click(object sender, RoutedEventArgs e)
        {

            // Open Serial Port
            SerialPort fpga_com_port = setup_serial_port();

            // Send START_OF_PACKET
            byte[] tx_data = { 0xD7, 0x8C, 0x1B, 0x74 };
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }
        private void Send_raw_data_Click(object sender, RoutedEventArgs e)
        {

            SerialPort fpga_com_port = setup_serial_port();

            // Send a Complex Test Sequence - lots of bad stuff and should only sync after END_OF_PACKET
            byte[] tx_data = {
                0x00, 0x01, 0x02, 0x03,
                0x04, 0x05, 0x07, 0x08
            };
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }

    private void Send_complex_test_1_Click(object sender, RoutedEventArgs e)
        {

            // Open Serial Port
            SerialPort fpga_com_port = setup_serial_port();

            // Send a Complex Test Sequence
            byte[] tx_data = {
            0x41, 0x6F, 0xDC, 0x1E,
            0xD7, 0x8C, 0x1B, 0x74,
            0x00, 0x01, 0x02, 0x03,
            0x04, 0x05, 0x07, 0x08 };
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }

        private void Send_complex_test_2_Click(object sender, RoutedEventArgs e)
        {

            // Open Serial Port
            SerialPort fpga_com_port = setup_serial_port();

            // Send a Complex Test Sequence - lots of bad stuff and should only sync after END_OF_PACKET
            byte[] tx_data = {
                0x41, 0x6F, 0xDC, 0x1E,
                0x41, 0x6F, 0xDC, 0x1E,
                0x00, 0x01, 0x02, 0x03,
                0x04, 0x05, 0x07, 0x08,
                0xD7, 0x8C, 0x1B, 0x74,
                0x00, 0x01, 0x02, 0x03,
                0x04, 0x05, 0x07, 0x08
            };
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }

        private void read_hwconfig_reg_Click(object sender, RoutedEventArgs e)
        {

            // Open Serial Port
            SerialPort fpga_com_port = setup_serial_port();

            // Send Test Sequences in little-endian
            byte[] tx_data = {
                // Reset - we transmit in little-endian
                0x1E, 0xDC, 0x6F, 0x41,
                // StartOfPacket
                0x74, 0x1B, 0x8C, 0xD7,
                // Command
                0x02, 0x00, 0x00, 0x00,
                // NumOfPayloadWords
                0x01, 0x00, 0x00, 0x00,
                // Payload - Just Address of HardwareConfigurationRegister
                0xF8, 0x00, 0x00, 0x00
            };
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }

        private void read_clkfreq_reg_Click(object sender, RoutedEventArgs e)
        {

            // Open Serial Port
            SerialPort fpga_com_port = setup_serial_port();

            // Send Test Sequences in little-endian
            byte[] tx_data = {
                // Reset - we transmit in little-endian
                0x1E, 0xDC, 0x6F, 0x41,
                // StartOfPacket
                0x74, 0x1B, 0x8C, 0xD7,
                // Command
                0x02, 0x00, 0x00, 0x00,
                // NumOfPayloadWords
                0x01, 0x00, 0x00, 0x00,
                // Payload - Just Address of HardwareConfigurationRegister
                0x81, 0x00, 0x00, 0x00
            };
            string tx_string = Send_serial_data(tx_data, fpga_com_port);
            // Rx Reply
            string rx_string = Read_serial_data(fpga_com_port);

            // Print
            Debug.WriteLine("Write: 0x" + tx_string);
            Debug.WriteLine("Read:  0x" + rx_string);
            Debug.WriteLine("Data Equal: " + tx_string.Equals(rx_string));

            // Close Serial Port
            close_serial_port(fpga_com_port);

        }

    }
}

Low-Cost Spatial Light Modulator Interface Board