Voice-Controlled Faucet

Introduction

One of the things that bothers me the most when you are cooking is having your hands full of grease or oil and going to wash and that the water is not at the right temperature. At that moment you have to manipulate the tap and you dirty everything. In addition to dirtying everything is a great health hazard.

The problem

You are massaging the turkey for Thanksgiving and you have oily hands. You approach the sink to wash your hands with your traditional automatic faucet, the water starts to come out and... Dammit, the water is too cold! Why can't I just tell the tap "hotter!"?

The solution

In the market there are solutions to this problem by means of taps that are activated by proximity sensors, however in these taps you cannot control the temperature of the water without manipulating it. I wanted to design a faucet with a proximity sensor that also allows you to control the level of water temperature by voice.

The SparkFun Artemis module is the world's first open source hardware RF module enabling both voice recognition and BLE. Therefore, why not use the Artemis module to design an automatic faucet with proximity sensor that responds by voice to control the temperature. This solution processes the voice offline, on the edge, without Amazon, Google, Apple or others listening to your conversations in the kitchen or in the bathroom. But this is not all... the Artemis module can also communicate with Bluetooth, so, let the tap can also be controlled by Bluetooth remotely.

Proposed design concept for the faucet

The system

Automatic Tap/Faucet

This automatic faucet combines several modules:

Inputmodules

• Microphone module: This module is in charge of listening to its surroundings, controlling the MEMS microphone, included in SparkFun RedBoard Artemis ATP board. Processes the PDM signal and gets audio samples.
• Proximity sensor module: Communicates by I2C with the SparkFun Proximity Sensor Breakout - 20cm, VCNL4040 (Qwiic) that is able to detect the presence of nearby objects without any physical contact. Emits a beam of infrared radiation and looks for changes in the field or return signal. The module resolves in which area of the faucet is the nearest object.
• BluetoothModule: This module controls the bluetooth communications. Allowing the system to communicate by Bluetooth low energy (BLE), low power wireless technology standard for personal area networks. (Included in SparkFun RedBoard Artemis ATP board).

Outputmodules

• Serial debug & reporting module: This module, if activated, communicates debug information sending serial data over the UART.
• LED array temperature level indicatormodule: this module displays selected temperature level lighting as many LEDs as the chosen temperature level.
• Solenoid valve control relays & solenoid valvesmodules: These two modules control the normally closed solenoid valves, to control water flow and water temperature by means of the activation of three relays.
• Blue led activity indicatormodule: The blue led activity indicator module controls the board blue led that blinks when the microphone module is active and stays steady on when bluetooth is activated.

Processing modules

• Microcontroller unit: The core of the system. It’s controlling the hardware that implements the device’s operation. The MCU receives inputs from the microphone, the proximity sensor and the bluetooth module and controls the the leds, the solenoid valves and coordinates the bluetooth communication.The microcontroller board is a SparkFun RedBoard Artemis ATP with Cortex-M4F, and ability to process TensorFlow TinyML.
• TensorFlow Lite interpreter: This module runs specially optimized TensorFlow models. Runs a 20 kilobyte neural network model to recognize keywords in speech. The project has a model that is trained to recognize "Up" and "Down" voice commands.

How it works

The system in action

• Snoopy licked Lucy and she is going to wash her hands.
• The system is in idle state, the blue LED is off.
• Lucy approaches the tap, the blue LED starts to blink and the tap listens to Lucy.
• Lucy is changing the water temperature level saying "Up" & "Down".
• The tap displays the level in the LED array display but does not activate the solenoid valves, the water does not come out.
• Lucy has already played in the region that does not activate the water outlet.
• Now she approaches the area that does activate the water. The blue LED continues blinking and listening to Lucy. Again Lucy gives orders to the tap, but this time the solenoid valves are operating depending on the temperature level chosen. The relay arrays shows the active relays lighting each led when active. Changes occur in the current consumed by the solenoid valves
• Finally Lucy is happy and has clean hands then separates from the tap. The current stops flowing through the solenoid valves. The tap closes the water outlet and returns to the idle state. The blue LED is now off.

Determining system state

The system state is determined by the distance detected by the proximity sensor to the nearest object. The tap has 4 zones defined depending on the distance detected by the proximity sensor:

• Idle zone, the sensor does not detect anything in its environment (<22 cm). The tap only serves the proximity sensor does not show the temperature level or operate the solenoid valves.
• Voice command only zone (distance between 18 and 22 cm). In this area the solenoid valves cannot be activated, there is no water coming out of the tap but the tap temperature level can be modified using voice commands (up & down).
• Normal tap use zone (distance between 1 cm and 18 cm. In this zone, tap water comes out, the solenoid valves are operated and the tap temperature level can also be modified using voice commands (Up & Down).
• Bluetooth zone (distance between 1 cm and 0 cm). The tap listens via bluetooth and temperature and tap opening can be controlled.

Faucet regions

Correspondencebetween distance range and tap regions

Statediagram

States

• Idle: the sensor does not detect anything in its environment (<22 cm). The tap only serves the proximity sensor does not show the temperature level or operate the solenoid valves.
• Closed tap and voice controlled level: the tap temperature level can be modified using voice commands.
• Open tap and void controlled level: tap water comes out, the solenoid valves are operated and the tap temperature level can also be modified using voice commands (Up & Down).
• Bluetooth controlled tap: The tap listens via bluetooth and temperature and tap opening can be controlled.

Transitions, based in proximity sensor estimated distance:

• A: Distance > 22 cm
• B: 18 cm < distance <= 22 cm
• C: 1 cm < distance <= 18 cm
• D: Distance <= 1 cm

Main loop

The states in the main loop. From source file automatic_faucet.ino

The distance to the object closest to the proximity sensor is recovered and the state in which the tap must be is computed. To avoid sudden changes between states, a debouncing technique is used.

void loop() {

    uint16_t proxValue = faucetProximitySensor->getProximity();
    State newState = getNewState(proxValue);

    actualState = debounceState(actualState, newState);

    switch (actualState) {
            case ONLY_LEVEL:
                processOnlyLevelState();
                break;
            case OPEN:
                processOpenState();
                break;
            case BLUETOOTH:
                processBluetoothState();
                break;
            default:
                processIdleState();
    }
}

Debouncing states

This routine is in charge of debouncing states. DEBOUNCE_COUNT indicates how many loop counts must pass to give a new state as valid.

#define DEBOUNCE_COUNT 10

State debounceState(const State actualState, const State newState){
    static int counter = 0;

    if (newState == actualState && counter > 0)
        counter--;

    if (newState != actualState)
        counter++; 

    if (counter >= DEBOUNCE_COUNT) {
        counter = 0;
        return newState;
    }

    return actualState;
}

Computing actual state

The new state is calculated with the distance to the object closest to the proximity sensor

enum State { IDLE = 0, ONLY_LEVEL = 1, OPEN = 2, BLUETOOTH = 3 };

State getNewState(const uint16_t proxValue)
{   
    if (proxValue > 5 && proxValue <= 20)
        return ONLY_LEVEL;

    if (proxValue > 20 && proxValue <= 400)
        return OPEN;

    if (proxValue > 400)
        return BLUETOOTH;
    
    return IDLE; // < 5 nothing detected in scope
}

Controlling Water Temperature

To control the water temperature, three solenoid valves are used. Two solenoid valves control the entry of cold water and a third controls the entry of hot water. As shown in the following table:

Each of the solenoid valves is controlled by a relay device connected to a digital output of the Sparkfun Artemis ATP development board

Faucet Controller

The relays are activated depending on the temperature level chosen as defined in the file faucet_controller.cpp

#define RELAY1 22
#define RELAY2 23
#define RELAY3 27

const int relays[RELAYS] = {RELAY1, RELAY2, RELAY3};

const int temperatures[LEVELS][RELAYS] = {
    // cold, cold, hot
        {0, 0, 0},    // level 0 tap is closed
        {1, 0, 0},    // level 1 cold
        {1, 1, 1},    // level 2 warm
        {1, 0, 1},    // level 3 hot
        {0, 0, 1}     // level 4 very hot
};

Activating and deactivating solenoid valves

Using the temperatures array in faucet_controller.cpp for activating and deactivating the relays and subsequently the solenoid valves.

void faucet_setValvesState(const int level) {
    for (int i = 0; i < RELAYS; i++) {
        if (temperatures[level][i]) {
            am_hal_gpio_output_clear(relays[i]);
        } else {
            am_hal_gpio_output_set(relays[i]);
        }
    }
}

Led Array Temperature Level Indicator

The Led Array Temperature Level Indicator displays selected temperature level lighting as many LEDs as the chosen temperature level.

LED array in code

Defining LEDS for the LED display array.

#define LED_LEVEL1 9
#define LED_LEVEL2 10
#define LED_LEVEL3 8
#define LED_LEVEL4 14

const int leds[LEVELS] = {LED_LEVEL1, LED_LEVEL2, LED_LEVEL3, LED_LEVEL4};

Displaying the level in file faucet_controller.cpp

void faucet_display_level(const int level){
    for (int i = 0; i < level; i++) {
        am_hal_gpio_output_set(leds[i]);
    }
    for (int i = level; i < LEDS; i++) {
        am_hal_gpio_output_clear(leds[i]);
    }
}

Lowering temperature

int faucet_lower_temperature_level(const int actuateValves) {
    if (actual_level > 0)
        actual_level--;

    faucet_display_level(actual_level);

    actuateValves ? faucet_set_valves_level(actual_level)
                    : faucet_valves_state_clear();

    return actual_level;
}

Raising temperature

int faucet_raise_temperature_level(const int actuateValves) {
    if (actual_level < 4)
        actual_level++;

    faucet_display_level(actual_level);

    actuateValves ? faucet_set_valves_level(actual_level)
                    : faucet_valves_state_clear();

    return actual_level;
}

Interpreting voice commands

The project has a model that is trained to recognize "Up" and "Down".

The neural network model used was built using the TensorFlow speech commands tutorial using Google Colaboratory, running Train up and down audio commands.ipynb in Google Colaboratory - an Junyper notebooks based system that presents a Python based notebook and abstracts the management of compute resources.

Application flow,

Commad responder

When a new command is listened it is processed in arduino_command_responder.cpp

if (is_new_command) {
    // Level up if 'up' was heard.
    if (found_command[0] == 'u' && found_command[1] == 'p') {
        faucetLevel = faucet_raise_temperature_level(activate_valves);
    }
    // Level down if 'down' was heard.
    if (found_command[0] == 'd' && found_command[1] == 'o') {
        faucetLevel = faucet_lower_temperature_level(activate_valves);
    }
    if (found_command[0] == 'u' && found_command[1] == 'n') {
        // unknown, log or do nothing
    }
}

Testing with water

Testing in more real conditions. The model does not work so well with the noise of water coming out of the tap. It is necessary to train it with audio samples taken in those same conditions.

Blooper

When things don't go as they should but you have a great time!

Controlling the faucet by Bluetooth

The main objective of the project is the voice control of the tap to control the temperature in a tap with proximity sensor.

Taking advantage of the possibilities of communication by bluetooth of the Sparkfun Artemis module, a very simple module has been developed that allows the control by Bluetooth communication similar to that carried out by voice.

The standard Immediate Alert service [org.bluetooth.service.immediate_alert] is used for simplicity.

GATT Service

• 00001802-0000-1000-8000-00805F9B34FB Immediate Alert

GATTCharacteristics

• 00002A00-0000-1000-8000-00805F9B34FB Device Name
• 00002A06-0000-1000-8000-00805F9B34FB Alert Level

The value of the characteristic shall be an unsigned 8 bit integer that has a fixed point exponent of 0. The Alert Level characteristic defines the level of alert, and is one of the following three values:

• Value 0, meaning “No Alert”
• Value 1, meaning “Mild Alert”
• Value 2, meaning “High Alert"

"Reinterpreting" the Alert Level GATT Characteristic

So that, when writing the value to the tap:

• "No Alert" [0x00] means close the tap;
• "Mild Alert" [0x01] means decrease the temperature level. Does the same function as the voice command "Down"
• and "High Alert" [0x02] means increase the temperature level. Does the same function as the voice command "Up"

And when reading:

• "No Alert" [0x00] tap closed;
• "Mild Alert" [0x01] cold or warm levels selected
• and "High Alert" [0x02] hot or very hot levels selected

It is used the Cordio Stack and Cordio Profiles that implements a BLE Host Stack as defined in the Bluetooth core specification [1] and GATT Profiles, respectively.

"Reinterpreting" the Alert Level GATT Characteristic

In nus_main.c the system responds to commands and actuates the tap:

static void tagAlert(uint8_t alert) {
    if (alert == CH_ALERT_LVL_NONE) {
        faucet_closeFaucet();
        AppUiAction(APP_UI_ALERT_CANCEL);
    } else if (alert == CH_ALERT_LVL_MILD) {
        faucet_lowerTemperatureLevel();
        AppUiAction(APP_UI_ALERT_LOW);
    } else if (alert == CH_ALERT_LVL_HIGH) {
        faucet_raiseTemperatureLevel();
        AppUiAction(APP_UI_ALERT_HIGH);
    }
}

Discovering the device:

The service is advertised as "Artemis Faucet"

GATTServices

It exposes the following services. You can control the faucet by writing to Immediate Alert characteristic, Alert Level GATT Characteristic

Checking theBluetooth Module

To activate the bluetooth module, you must lower the tab that covers the proximity sensor.

The hardware

Components

Solenoid valves

The solenoid valves physically starts and stops the water flow. When electricity is applied the coil generates a magnetic field causing the plunger to be attracted to a stop. The stop is a magnetic piece affixed to the top of the plunger guide. In order to remain in this "open" position, the coil must receive constant electrical power to maintain the magnetic field holding the plunger in the open position.

If the valve remains open for long periods of time or if the valve is battery powered and must conserve as much energy as possible a better solution will be to use latching solenoid valves.

SparkFun Proximity Sensor Breakout

The SparkFun Proximity Sensor Breakout is a simple IR presence and ambient light sensor utilizing the VCNL4040. This type of sensor is often seen in automatic towel and soap dispensers or automatic faucets. With the ability to detect objects qualitatively up to 20cm away, you will be able to detect if something is there and if it is closer or further away since the last reading. Utilizing SparkFun Qwiic system, no soldering is required to connect it to the rest of your system.

SparkFun RedBoard Artemis ATP

The SparkFun RedBoard Artemis ATP features 48 GPIO and Arduino mega footprint. Includes a modern USB-C connector that makes programming easy and a Qwiic connector to make I2C easy. It is fully compatible with SparkFun's Arduino core and can be programmed easily under the Arduino IDE. This Artemis ATP board consists of a digital MEMS microphone that functions with always-on voice commands with TensorFlow and machine learning.

It runs at 48MHz with a 96MHz turbo mode and with Bluetooth to boot. This Artemis ATP board has 10 ADC channels with 14-bit precision with up to 2.67 million samples per second effective continuous, multi-slot sampling rate. It also features 2 channel differential ADC, 2 UARTs, 6 I 2 C buses, and 6 SPI buses. This Artemis ATP board operates at 6uA / MHz that is less than 5mW at full operation. It is designed with a built-in BLE radio and 1M Flash / 384k RAM.

Relays modules

This relay module allows to switch between AC 250V / 10A or DC30V 10A using a microcontroller. It has simple handling through a digital output. This module can be activated with 5V or 3.3V digital outputs.

Pinout

Construction

Materials

Tools

Making the led array indicator

The array shares GND for the four LEDs. The proximity sensor is attached to the LED array PCB with two plastic standoffs.

Front view

Rear view

Making the relays array

The array shares VCC and GND for the three relays.

Front view

Rear view

Putting all together

Connecting the solenoid valves to the relays. Each relays controls one solenoid valve. One for hot water and the other two for cold water.

Software Installation

Three Github repositories are included:

• The complete Automatic faucet project
• Artemis ATP Speech Command Level Controller, a project to check the TensorFlow Lite model generated for the Tap
• Artemis ATP TensorFlow Micro Speech, a tutorial on how to use TensorFlow models and the integrated microphone of the ATP board.

The code in the repository is for use and execution within the Arduino development environment or Microsoft VS Code with the Arduino extension. This section details the steps required to setup Arduino.

Install Arduino

Arduino is available for a variety of platforms. To ensure compatibility with the demos in this repository, the latest version should be installed.

The Arduino application is available for a variety of platforms and is available online from Arduino. Download the application from the Arduino website using this link.

Install the TensorFlowLite Library

The project utilize the TensorFlowLight Arduino library, which is installed using the Arduino Library Manager.

To install this library, use the following steps:

• In Arduino, select the "Manage Libraries..." menu item. Tools > Manage Libraries...
• In the Library Manager interface, search for tensorflow
• Select the library Arduino_TensorFlowLite by TensorFlow Authors
• Select the non-precompiledversion of the library. This is important!
• Select the Install button to install the library

Install the SparkFun Boards Package

Load the SparkFun Boards package into the Arduino Board Manger.

To install package, use the following steps:

• In Arduino, open the Preferences menu item. File > Preferences, (macOS) Arduino > Preferences
• Add the following path to the Additional Boards Manager URLs: path in preferences.https://raw.githubusercontent.com/sparkfun/Arduino_Boards/master/IDE_Board_Manager/package_sparkfun_index.json
• Select the OK button to save the preferences.

Install the Sparkfun Artemis Boards Package

Once the location of the SparkFun boards package is set in the preferences, the board definition package for the SparkFun Apollo3 boards must be installed.

To install package, use the following steps:

• In Arduino, open the Preferences menu item. Tools > Board "..." > Manage Boards...
• Search for SparkFun
• Select the SparkFun Apollo3 Boards package. The latest version is OK.
• Close the dialog Select the SparkFun Edge Board. Tools > Boards "..." > SparkFun RedBoard Artemis ATP
• Set the Bootloader to the SparkFun Variable Loader. ***Tools > Bootloader "..." > SparkFun Variable Loader..."

Install the Qwiic Proximity Sensor (VCNL4040) library

You can grab it from the Arduino Library Manager or from the GitHub repository

The sensor has a single slave address 0x60 (HEX) of 7-bit addressing, following I2C protocol.

With the Qwiic connector system, assembling the hardware is simple. All you need to do is connect your Qwiic Proximity Sensor to the RedBoard Qwiic with a Qwiic cable.

Configure the Software

• Install and setup Arduino as described above.
• Download (or clone) the repo to favorite location
• Open Arduino
• Open by clicking on the.ino file
• Select the SparkFun RedBoard Artemis ATP board (Tools->Board under 'SparkFun Apollo3')
• Windows - COM Port
• Linux - /dev/ttyUSB*, where * is a number. NOTE: Permissions on this device needs to include user read and write (chmod 666)
• macOS - /dev/cu.usbserial*
• Change the Bootloader from Ambiq Secure Bootloader (Default) to SparkFun Variable Loader (Enable w/ Artemis Bootloader) (Tools->Bootloader)
• Change the SVL Baud Rate from 921600 to 230400

Upload

• Compile with the Verify button (check mark symbol)
• Upload with the Upload button (arrow symbol)
• If uploading fails try lowering the bootloader baud rate
• or unplugging the board for several seconds
• Note: You will see some warnings from the compiler, these can be ignored.
• Note: The bootloader attempts to upload for 3 times. If all three attempts fail, try adjusting the baud rate.

Serial Monitor

• Open the Serial Monitor by clicking on the magnifying glass icon (top right) or (Tools->Serial Monitor)

Checking the development environment

Making a Simple Speech Command Level Controller

The project has a model that is trained to recognize "Up" and "Down". The neural network model used was built using the TensorFlow speech commands tutorial using Google Colaboratory, running Train up and down audio commands.ipynb In Google Colaboratory - an Jypter notebooks based system that presents a Python based notebook and abstracts the management of compute resources.

Next steps

Another interesting application for this tap would be the detection of open faucets or leaks by detecting and identifying the dripping noise or water coming out of the tap while in idle state.

Lessons learned and thanks

Initially my project was going to be another "Artemis Says", a Simon Says clone, but the OLED display of the kit arrived broken.

Sparkfun offered to send me a replacement but, fortunately :), it didn't arrive on time. So this is my 'plan B' of which I am very satisfied.

The potential of this small MCU and the ease of setting it up and programming it is incredible. Good job, Sparkfun!

More than 20 different people have downloaded my tutorial Artemis ATP Tensorflow Micro Speech from GitHub and I hope it has helped them with their projects.

https://www.hackster.io/javagoza/artemis-atp-wake-word-detection-d95f08

Hackster.io is a great community to learn and share.

And finally another project I've made using the SparkFun Proximity Sensor Breakout - 20cm, VCNL4040 (Qwiic) from the kit:

https://www.element14.com/community/groups/azuresphere/blog/2019/11/28/i-was-here

In this one I've ported the SparkFun VCNL4040 library to the Azure Sphere OS

https://github.com/javagoza/I-Was-There/blob/master/IWT_HighLevelApp/vcnl4040.c