Turning a New Page in AI

While there is still plenty of room for improvement, generative artificial intelligence applications have really come into their own in the past few years. Large language models (LLMs) have captured the public’s interest for their uncanny ability to summarize text, understand natural language, and create content. Similarly, text-to-image generators have fascinated us with the way that they can turn our text-based requests into photorealistic images or fantastic science fiction-worthy scenes.

As previously mentioned, these tools do still leave much to be desired at times. LLMs have become notorious for hallucinating and inventing all manner of falsehoods that they present as true. Image generators also have their share of fun, perhaps drawing horses with five legs or making other such odd errors that even a toddler would know to avoid.

These shortcomings are not always a bad thing, however. When creativity is valued over accuracy, hallucinations may be useful (listen to a Beatles album from the late 60s if you need proof of that). An engineer by the name of Thomas Valadez took advantage of that creativity to build an interesting device called the Storybook Generator. This device uses an LLM and an image generator to produce single-page scenes from a storybook that never existed and display them on an E Ink display.

The project only requires a few components — a Raspberry Pi 5 8GB and an Inky Impression 5.7 inch E Ink display. On the software side, Ollama was leveraged to make deployment of an LLM to the Raspberry Pi a simple procedure. By issuing just a few commands, it was possible to have a Google Gemma 7B model installed and responding to prompts.

For image generation, Stable Diffusion XL Turbo 1.0 was selected, which is a much bigger ask for the resource-limited hardware platform that was chosen. However, with the help of Onnxstream (which we covered last summer), the memory requirements can be slashed dramatically.

By carefully crafting prompts to the LLM and the image generator, an illustrated storybook page can be produced and displayed on the E Ink screen. Due to the nature of these displays, it can remain on throughout the day without consuming energy, offering entertainment and inspiration.

Running these cutting-edge algorithms on a Raspberry Pi is not exactly fast — generating a new page takes about 5 minutes of processing time. But for this particular application, that is not really a problem. New images appear periodically throughout the day, and the exact timing is of little consequence.

If you have a Raspberry Pi 5 and an E Ink display, this looks like an easy and fun build. Take a look at the GitHub repository for setup instructions and to grab the source code.