Software apps and online services
Each Biomaker team receives a 4D Systems µLCD-32DT-AR Arduino Display Module Pack, which includes a Gen4 µLCD-32DT 3.2" LCD display with resistive touchscreen, a 4D Arduino adaptor shield and 5 way interface cable. The LCD display can be hooked up to the Arduino via a serial port. It is possible to build sophisticated user interfaces for Arduino-based instrumentation using graphical tools.
The Arduino adapter shield allows the µLCD display to be interfaced directly with the Biomaker starter kit without any wiring hassles. The µLCD-32DT is an intelligent display. It possess a dedicated microcontroller and storage which handles a range of functions for interactions and graphical display operations. 4D Systems provides a free Windows-based development environment (4D Workshop - see below) that allows non-programmers to build sophisticated user interfaces, using a drag-and-drop graphical interface. A wide range of interactive widgets can be arranged in a series of forms, customised by setting parameters, and downloaded to the programmable touchscreen. An Arduino microcontroller (or Raspberry Pi or computer) can communicate with the customised touchscreen by simple serial commands, and the screen manages all of the hard work of handling graphics and touchscreen interactivity.
Members of the XOD community have created XOD nodes and libraries that allow simple communication with the programmable screens. They provide a comprehensive range of serial commands to send data for display on the LCD display, and to receive touchscreen based instructions, such as button presses, slider values, etc. The 4D Systems screens handle all interactivity and graphical display - without needing explicitly code these. We provide a 3.2" touchscreen for Biomaker, but a wide range of other screen devices are available, which use the same development environment.
The prospect of "gluing" easily built and easily customised user interfaces to simple Arduino hardware with XOD based graphical programming opens up new opportunities for scientists, engineers and inventors. In order to introduce these powerful devices to new users in simple steps, we have pre-programmed touchscreens with a set of multi-screen graphical widgets, and use these to demonstrate:
- how to receive and display messages from hardware on the Biomaker Rich UNO R3 board.
- how to send information to the Biomaker Rich UNO R3 board after user interaction with the touchscreen.
Some additional detail and technical background is provided below, and there are relevant manuals, data sheets, application notes and instructional videos available via the 4D Systems website (https://4dsystems.com.au).
We have produced a set of interface elements for downloading to a Gen4 µLCD-32DT 3.2" LCD display. The layout of elements was composed in 4D Workshop and downloaded, in a two-steps, to the memory and µSD card of each display. The screens (called Forms in 4D Workshop) can each contain a numer of interactive elements, and the five Forms are shown below. The 4D Workshop code for this interface will be available to download - see below.
The names of all of the widgets are shown - these are needed to allow the XOD code to send and receive messages with the correct elements.
1. Install the XOD libraries for communication with 4DS touchscreens.
2. Configure and connect the 4DS touchscreen and Arduino shield to the Rich UNO R3 board
3. Use the Rich UNO R3 onboard hardware (see Biomaker Tutorial 3) to send parameters for display on the touchscreen.
4. Interact with touchscreen devices to control hardware on the Rich UNO R3 board.
Load copies of two XOD libraries that allow Arduino control of custom interfaces at https://xod.io/libs/gabbapeople/4d-ulcd/ and bradzilla84/visi-genie-extra-library (install both), and please provide feedback to the developers. These tools are hugely enabling - providing a code-free development environment for building sophisticated user interfaces for low cost, DIY instrumentation.
You will also use code from other XOD libraries, which will be useful to install. Cesar Sosa, based in Argentina, has customised code for use with the Biomaker Rich UNO R3 board (XOD library: cesars/rich-uno-r3), which we use to control the onboard TM1637 4-digit 7-segment display - and the library contains other useful functions.
The Arduino device and the touchscreen are connected via the Arduino shield, and communication is established by setting up a serial port connection through the UART node. Links to documentation can be found at 4D Systems (https://4dsystems.com.au/products/4d-intelligent-hmi-display-modules/arduino-compatible-kits/gen4-ulcd-32dt-ar) - key links are listed at the bottom of this page.
Points to note:
(i) There are two types of connections required. First, temporary connection via a USB port(s) to programme the 4DS screen and download graphics data to the µSD card - from a computer running the 4D Workshop development environment. Second, for connection to a serial communication port on the Arduino microcontroller, and the custom Arduino shield is provided for this.
(ii) When connecting screens and adapter via the provided ribbon cable, the blue marking on the cable should be positioned on the rear of the cable.
(iii) The default serial port for communication is Com0, which causes conflict with the only hardware serial port (D1, D2) on Arduino UNO compatible boards like the Rich UNO R3 (D2, D3). Shift jumpers J3 and J4 to select Com1, which will rely on software-based serial communications, but means that you won't have to unplug the screen everytime the Arduino is being programmed! Details can be found at: https://4dsystems.com.au/mwdownloads/download/link/id/85/
(iv) The use of the Arduino ports D2, D3 and D4 (for the default reset signal) requires you to switch off the the relevant DIP switches on the Rich UNO R3. This prevents use of the two left-most touch switches and the IR transmitter.
ViSi-Genie is used to design the user interface for the specific display model. The layout can be tweaked interactively in the development software. When finished, the interface code is downloaded to the display controller via its serial port. GUI interface features can be tested by interacting with the display. ViSi-Genie also provides debugging features for serial communication. When ready, the programmed display can be connected to an Arduino board via serial ports. The Arduino must then be programmed to communicate with the different graphical screen elements.
Use the xod/common-hardware/pot node, connected to port A0 to generate a user-controlled output, that will vary from 0 to 1. Connect the output to a xod/math/map node to convert the range to 0-100. A xod/debug/watch node can be used to check that the output is being generated correctly.
Add nodes from the cesars/rich-one-r3/00-list-readme-rich-one-r3 XOD library entry for control of the TM1637 4-digit display on the Rich UNO R3 board, as shown. This allows direct display of parameter values, as a reference, before we attempt to plug in the touchscreen display. The nodes include tm1637-r and set-tm1637-r, which allow control of the address and settings for the 4-digit display; data-display-tm1637-r and multiplexing-tm1637-r which format numbers for display, and the display node itself, display-dig-tm1637-r. The mapped output from the pot node is fed directly to the display and updated regularly.
Complicated "spaghetti-like" connections between multiple nodes can be minimised by the use of bus connections. An output value can be connected to a xod/patch-nodes/to-bus node, and input values can be obtained by connecting a xod/patch-nodes/from-bus node to a given XOD port. Each bus identity is configured by a common distinct name that is assigned by the user. For example, below the mapped output of the potentiometer is shared across the bus called "POT".
First, the xod/uart/soft-uart node is added to allow two-way communication between the Arduino and touchscreen board, and defines the use of ports D2 and D3 and a data rate of 9600 baud. This node is attached to bradzilla84/visi-genie-extra-library/int-filtered node (from the gabbapeople/4d-ulcd/4d-ulcd library) and helps manage communication with the touchscreen. The output of this node is shared via the SER bus.
For this tutorial, we have programmed the touchscreen with a series of forms (screen pages) labelled 0-4. The buttons in the bottom right hand corner of the screens are set up to toggle through the different screens in order. Each of the forms contains a number of interface elements (shown above) that can be read or written to by the Arduino microcontroller. For example, Form0 contains the graphical element Coolgauge0. Coolgauge is a type of display element defined in the 4D Workshop Genie environment. Every instance of this element will have a numerical identifier, 0 in this case - so that individual interface elements can be identified umambiguously.
The modified patch contains the gabbapeople/4d-ulcd/write-cool-gauge node. This node allows the simple transfer of a value to the nominated Coolgauge object (identified by the IDX parameter). In this example, the mapped value of the potentiometer is sent to be displayed by the gauge on Form0 - the default front page of the programmed display.
In addition, the xod/core/pulse-on-change has been added. This node produces a trigger whenever the value of the shared POT parameter changes. The trigger is shared on the PULSE bus, and used to initiate an update by the write-cool-gauge node. The intelligent touchscreen display handles all graphic animation and user interaction - so it is simple to build complex custom user interfaces.
The patch has been further modified to share the mapped potentiometer values with three other interface elements on Form1. These are: Meter0, Thermometer0 and Leddigits1 (see key above). The SER, POT and PULSE parameters are all shared via their respective bus, and sent to the touchscreen by the nodes write-meter, write-led-digits and write-thermometer nodes, respectively. Toggle to the Form1 screen to see dynamic updates of the potentiometer value - which should also be mirrored on the Arduino 4-digit display.
A number of other interface elements, including switches, knobs, buttons, sliders and trackbars are provided in the demonstration programme for the touchscreen. The relevant nodes can be found in the two XOD libraries that you should install. Take care to correctly address the interface elements, and you can read and write information to each device.
Interface elements that produce outputs - such as switches and trackbars and sliders can be read from inside XOD. Thus the touchscreen can be used to interact with programs. This is a simple matter of adding XOD nodes that allow reading of particular interface elements, and the out puts can be fed into the XOD patch.
In the example below (12-widget-to-buzzer), two touchscreen buttons (4Dbutton3 and 4Dbutton4) from Form2 are read, and the outputs are used to switch on--off an onboard buzzer and LED, respectively. Further Trackbar1, also on Form2, is read - and the output is passed to a map node, re-mapped between values 0 and 5000, and used to control the pitch of the buzzer. This is an example of how instrumentation can be controlled by interaction with the touchscreen. There are a wide range of other interface elements to experiment with! All of the user interactions, graphics and animations are handled behind the scene by the 4D Systems software and hardware - so it is trivial to program and prototype new interface designs. XOD allows the simple interfacing of additional sensors and display devices.
Below are some more details of the wide range of other interface objects that are provided in the 4D Workshop environment
The 4D-µLCD nodes can be used read and write data to a large number of screen elements. For example, inputs from switches and knobs can be used to animate on-screen indicators and gauges. Most of the animated interactions are handled directly by the display processor and the ViSi-Genie programmed elements (for example, graphical responses to touching an on-screen switch).
This task is made easier by (i) a limited number of precomposed user interface elements, and and conserved communication formats and addresses, and (ii) the availability of the XOD libraries for code-free programming of 4D Systems touchscreens - thanks to Max Danilin and Victor Nakoryakov, who are play a key role in the development of XOD. They haved helped with the development of XOD nodes for touchscreen support (https://forum.xod.io/t/visi-genie-library-to-work-with-4d-ulcd-displays/1625) - to allow simple programming of sophisticated interfaces and communication with Arduino-controlled hardware, using XOD nodes. (Check out https://youtu.be/9Nm_3ucTNcM for a video summary of the type of outcomes one can expect). The draft XOD library can be found at: https://xod.io/libs/gabbapeople/4d-ulcd/. It provides a library to work with gen4-µLCD-DT display modules by 4D Systems. This library provides graphical nodes for basic operations with common ViSi-Genie objects of the 4D Systems workshop IDE - and allows simple programming of user interface interactions.
- read-dip-switch: Reads a value from a genie "DIP SWITCH" object
- read-knob: Reads a value from a genie "KNOB" object
- read-rocker-switch: Reads a value from a genie "ROCKER SWITCH" object
- read-rotary-switch: Reads a value from a genie "ROTARY SWITCH" object
- read-slider: Reads a value from a genie "SLIDER" object
- read-trackbar: Reads a value from a genie "TRACKBAR" object
- read-winbutton: Reads a value from a genie "WINBUTTON" object
- write-angular-meter: Writes an integer value to a genie "ANGULAR METER" object
- write-cool-gauge: Writes an integer value to a genie "COOL GAUGE" object
- write-custom-digits: Writes an integer value to a genie "CUSTOM DIGITS" object
- write-led: Writes an integer value to a genie "LED" object
- write-led-digits: Writes an integer value to a genie "LED DIGITS" object
- write-meter: Writes an integer value to a genie "METER" object
- write-tank: Writes an integer value to a genie "TANK" object
- write-user-led: Writes an integer value to a genie "USER LED" object
- init: Initializes a 4D uLCD device with custom settings
- Contrast: Display on or off (uLCD-43P/PT/PCT Brightness 0-15)
- Dip switch: Read and Write
- Knob: Read and Write
- Rocker switch: Read and Write
- Rotary switch: Read and Write
- Slider: Read and Write
- Trackbar: Read and Write
- Winbutton: Read and Write
- Angular meter: Read and Write
- Coolgauge: Read and Write
- Custom digits: Read and Write
- Form: Write
- Gauge: Read and Write
- Image: Read and Write
- Keyboard: Read
- Led: Read and Write
- Led digits: Read and Write
- Meter: Read and Write
- Strings: Pre-Read, Pre-Write, Dynamically Created ASCII
- Thermometer: Read and Write
- User led: Read and Write
- Video: Read and Write
- Sound: Read, Play, Volume, Pause, Continue, Stop
- Timer: Write
Navigate to: https://www.hackster.io/wayland2/an-xod-library-for-the-bno055-absolute-orientation-sensor-52e877 - to see a documented project from Matt Wayland that describes use of the 4D Systems touch screen as an output device in XOD.
The 4D Systems touchscreen display can be hooked up to the Arduino via a serial port. It is now possible to build sophisticated user interfaces for Arduino-based instrumentation using graphical tools. Here’s a guide for how to go about this:
1. Program the touchscreen display
- Plan your scheme for hardware connections and lay out a draft of the onscreen indicators and control elements that you want to use.
- Load the 4D Systems IDE on a Windows PC, or Windows emulator on a Mac (Parallels works well). The software environment provides ViSi-Genie, a wysiwyg tool for layout of user interface tools.
- ViSi-Genie provides a wide range of interactive user interface elements for display of gauges, meters, indicators and input devices such as switches, knobs, sliders and buttons.
- After laying out the screen display, (i) the interactive code can be downloaded to the flash memory in the programmable display via a 4DS adapter, and (ii) the graphical elements can be downloaded directly to a µSD memory card that plugs into the display.
2. Build interaction between the Arduino hardware and the customised display
- There is now a XOD library that allows graphical programming of Arduino hardware for two-way communication with screen graphical elements through a serial port.
- Use the XOD graphical programming environment to build interactions between the Arduino-connected hardware and screen widgets.
- Test interaction between the Arduino hardware and 4DS touchscreen through the XOD development environment.
4D Systems provides four different environments for programming the touchscreen - we recommend use of the ViSi-Genie interface for code-free layout of custom user interfaces. Obtain from: (https://www.4dsystems.com.au/product/4D_Workshop_4_IDE/downloads). The basic version is free.
4D Systems provides a free Windows-compatible software environment for programming the touchscreen displays. Workshop4 includes four development environments, for the user to choose based on application requirements or user skill level. ViSi-Genie is an advanced environment that doesn't require any text-based coding, it is done automatically for you. The software provides a toolbar filled with graphical widgets. These can be simply dragged to lay out the prototype interface with the objects you want, set the events to drive them and the code is written for you automatically.
ViSi-Genie is a drag-and-drop rapid development tool for designing and building graphic user interface on 4D Labs processor-based displays. It provides an easy method for designing complex graphics user interface (GUI) applications without any coding. A simple GUI application to be ‘designed’ from scratch in literally seconds. ViSi-Genie does all the background coding, with no programming language to learn.
Above:Screen objects can be interactively resized, positioned and aligned in ViSi-Genie, before being downloaded to the display.
Pick and choose the relevant objects to place on a virtual display. The full animation of the objects is done under-the-hood, such as pressing a button or moving the thumb of the slider. Each object has parameters which can be set, and configurable events to animate and drive other objects or communicate with an external host. Simply place an object on the screen, position and size it to suit, set the parameters such as colour, range, text, and finally select the event you wish the object to be associated with, it is that simple. Objects are classified in three different groups:
- INPUT OBJECTS, as a button or a keyboard,
- OUTPUT OBJECTS, as a gauge or a meter, and
- COMBINED OBJECTS or INPUT/OUTPUT OBJECTS, as a slider which acts as both an input and an output.
In seconds you can transform a blank display into a fully animated GUI with moving meters, animated press and release buttons, and much more. The assembled code can be directly downloaded to a 4D Systems display to check the screen display. (Note: compiled code is downloaded to the display ViSi-Genie, and stored graphic elements are separately downloaded to a µSD memory card in a semi-automated process). ViSi-Genie provides debugging tools to interrogate communication streams to/from the programmed display. The ViSi-Genie environment runs under Windows, but can be well implemented under Mac OS X using an emulator like Parallels.
Above: examples of button/switch input widgets that can be added to a display, to create a custom user interface.
Lists of the types of elements that can be added to the 4D Systems displays from the ViSi-Genie programming environment.
After a user interface has been developed in ViSi-Genie, it must be downloaded to the 4D Systems display in a two-step semi-automated process. Code is loaded onto non-volatile memory on the display board via its serial port (while it is disconnected from the Arduino). Second, graphical interface data is loaded onto the display's µSD card, using a card writer attached to the ViSi-Genie host computer. The µSD card is plugged back into the 4D Systems display, and display plugged back into the Arduino via the dedicated shield.
4D Systems Application Note: Connecting a 4D display to an Arduino host
This application note shows how to create a ViSi Genie program and how to use the ViSi Genie library for the Arduino IDE. A simple project is described. This consists of a 4D Picaso module displaying six objects – a LED digits, a slider, a cool gauge, a string, a user LED, and a static text (label). To recreate the application described in this demo, the user first creates a ViSi Genie program in the 4D Workshop IDE and downloads it to a 4D display module. In this example, the Arduino host is programmed using the Arduino IDE. The Arduino and display communicate via a serial port. The Arduino programme loop must be configured to receive commands from interactive controls on the display, and to send relevant updates.(https://www.4dsystems.com.au/appnote/4D-AN-00017/)
Once a graphical interface has been programmed and downloaded to the display using ViSi-Genie-Genie in the 4D Systems Workshop4 environment, Arduino hardware can be used to communicate with individual interface elements. Communication takes place through a serial port, where information is read from, or written to display objects through a series of commands. The commands take the general format of:
Format for serial communication between the programmable display and the host device, which could be a Biomaker Arduino, computer or Raspberry Pi. Each command is followed by an identifier for the target GUI widget, parameters and checksum.
1. No Comms, Arduino will not talk to Display
a) Check your Baud Rates are the same in your Arduino, and in the Display.
b) Check you are using the correct Serial Port on the Arduino, and on the Display, so they match where you have actually plugged them into the two devices.
c) Ensure you are not doing any Serial.prints or similar using the same Serial Port as you are using to talk to the Display. Depending how many you do and how often you do them will either make the comms not work at all, be extremely intermittent, or just cause slowness of the whole system. You cannot do Serial.print statements to the same serial port as you use to talk to the display. Big no-no.
d) Ensure you have the Reset Code in your sketch, for the Arduino to reset the Display Module at startup, so the display and Arduino are in sync and both are in a known state. Without this code, your program may not work at all, the display may appear to be not communicating, or you may have seemingly slow comms. Please refer to the Demo which comes with the Genie Arduino library, and implement the Reset logic in your sketch.
e) If you have a large program in WS4, you may need to increase the delay after your reset code in your setup() routine, from 3500ms to 5000ms or even higher. If the Arduino starts talking to the Display before the Display is ready, then it can flood the library and get the two out of sync and cause all sorts of grief.
f) Depending on which Arduino you are using, will determine how many serial ports (UARTs) it has available. Arduinos like the Uno only have one Hardware Serial UART, and it is shared with the USB Programming port of the Arduino. So when you are programming the Arduino and you also have your display connected to Serial (Serial0), then you must unplug your display from the Uno in order to program it, or you will get a conflict and programming will fail. If you have a Mega2560 for example, it has 4 serial ports. By default the 4D Arduino Adaptor Shield uses Serial0, which is what the USB for programming uses on the Mega also, so if you want to change it to another Serial port, such as Serial1, Serial2 etc, then you can remove the jumpers on the 4D Arduino Adaptor Shield and place a jumper wire from the center pin of that header, into the desired serial pins on the Arduino headers. Please refer to our Application Notes for more information. There are so many Arduinos and combinations available, the Shield uses the Serial Port which suits the majority, however with a little creativity, it is possible to change the Serial Port quite easily, even to use Software Serial.
g) Ensure you are only talking to objects which exist. You cannot have Arduino code talking to String0 for example, if String0 does not exist in your WS4 application. This will result in data being written to RAM on the display which is either unallocated, or allocated to something else, resulting in failed comms, corruption or weird behavior. Ensure you only write to objects which you have on the display.
2. Corrupt Graphics on the display, or Red X's through gauges, Display restarting etc.
a) Corrupt graphics are commonly caused by getting the Arduino to write to objects/widgets which do not exist on the Displays application. For example, if you are doing a genie.WriteObject to a LED Digit, say LED Digit 3, and you only have LED Digits 0, 1 and 2 in your application, then this will cause problems. You cannot write to something which doesn't exist. This can cause your entire program to fail!
b) Red X's are caused by you writing something from the Arduino to the display, which is outside the range of what that object is configured to accept. For example, you are writing the value of 123 to an LED Digit which is only 2 digits long and can therefore only accept a value from 0 to 99.
c) Your brand of microSD Card. Not all microSD cards are created equal. Some available on the market are cheap copies, and some are just not compatible with our display modules. Our modules (ALL of our modules) require microSD cards which are SPI compatible. This is why we sell microSD cards ourselves, as we know the manufacturer and buy direct from them. There are a number of brands of cards which just do not work, one in particular is Transcend. For some reason, virtually all Transcend cards do not work in our modules, and if they do they may not work for long. There are other brands also. Please stick with known good quality cards, or buy from us directly as our cards work.
d) If you are using a larger display, such as a 4.3" or a 7.0", you need to ensure you are supplying enough power to the display. The 7.0" draws around 500mA of current at 5V just by itself. This is right on the limit of what most computers can output. If you have an Arduino connected to the same USB port, or you are trying to supply power to the Display via the Arduino using USB, then you are likely to run into problems of the 7.0" display restarting over and over. Either supply power separately to the display using the Arduino Adaptor Shield H2 (refer to its Datasheet), or supply power to the Arduino via its Barrel Jack. Take note that if you are going to use the Barrel Jack on the Arduino, don't put in a voltage over 12V as the Linear Regulator on the Arduino will have to dissipate all that heat and it will likely shut down. 12V may even be too high in some cases. Do the maths.
3. Comms are slow, why?
a) Check question 1 above
b) Do you have any delays in your Arduino code? If you do, remove them. Delays are Blocking and prevent the display from talking to the Arduino as the Arduino is sitting in a paused state and not listening to anything anyone says. If the display is trying to send information to the Arduino when the Arduino is in a delay, the information will be lost. Depending how long the delay is, it can prevent the application working at all, or can cause the system to crash due to comms getting out of sync. If you need a delay, consider using the method descibed in the 'Blink without delay' example that comes with the Arduino.
c) is the genie.DoEvents() function allowed to run every single loop, without restriction? This is the doorway for the Arduino to talk to the Display. This function must be allowed to run each loop so the library can then be relieved of its buffered communications from the display. Please, don't put delays in the loop and don't put this genie.DoEvents in some conditional statement unless you 100% know what you are doing.
d) Try a faster baud rate, such as 200000 Baud, on both the Display and the Arduino. Depending how much information you are sending between the display and the Arduino, may warrant a faster baud rate.
e) see g) above in section 1.
4D Systems ViSi Genie User Guide: (https://4dsystems.com.au/mwdownloads/download/link/id/26/)
4D Systems ViSi-Genie Reference Manual: (https://4dsystems.com.au/mwdownloads/download/link/id/27/)
Arduino IDE software library: (https://github.com/4dsystems/ViSi-Genie-Arduino-Library)
Demonstration: This demo creates a touchscreen calculator
Code: (https://github.com/4dsystems/ViSi-Genie-Arduino-Demo-Calculator/) Video: (http://www.youtube.com/watch?v=KFdDmTqGhK4)
Demonstration: This demo controls a tri-coloured LED along with displaying images from the SD card on the display in a slide show.
Arduino IDE software library:4D Systems provides an Arduino IDE library that allows the use of programmed displays with Arduino boards programmed with the standard IDE. (https://github.com/4dsystems/ViSi-Genie-Arduino-Library)
This IDE library provides high level functions for the Arduino, to simplify communication with 4D Systems modules when using a module configured with ViSi-Genie. Inside the library are 3 example sketches, to assist with getting started using this library. Inside is also a ViSi-Genie Workshop4 project, which can be used on a range of 4D Systems displays (designed for a uLCD-32DT). It illustrates how to use some of the commands in the library include Read Object, Write Object, Reported Messages, Write Contrast and Write String. There are well-documented examples available from 4D Systems.
XOD software libraries:
Review: 4D Systems Display Starter Kit
4D Systems Gen4 µLCD 3.2” resistive touch screen: Current Price: around £50/$70 per kit (e.g. RS online: https://uk.rs-online.com/web/p/lcd-colour-displays/12580202)
International distributors: https://www.4dsystems.com.au/distributors