Back in 2020, I build an ESP32 based project to show, every morning, the cover page of the New York Times on a e-paper display.
This project has been working well for almost 5 years, and I am (almost) surprised that it was even useful, i.e. people in my family were actually looking at the screen from time to time to check on the US news.
When I saw this very slick e-paper display from M5stack, I said to myself, "hell, let's do it again, this time with way more functionality"
This new project is available on github and on M5 stack Project Zone. It is written in micro-python (using M5stack's IDE "UIflow2").
News from US, EU, ChinaThe original concept was to grab the cover page of the New York Times, which the newspaper (kindly) provides, every day, as a pdf.
The M5PaperS3 has ample storage capabilities (flash and SD), so this new project also grabs cover from European and Asian's newspapers.
- I am French so I picked Liberation, a french paper famous for its cover page.
- I also went for the New Yorker. Its cover have been famous since the 50's.
- Finally, I thought the China Daily would provide a world wide coverage.
Every so often (every 4h, 8h), the M5PaperS3 wakes up, grabs the latest set of covers available online, stores them locally, displays one cover and shuts the power down.
This entire process takes about 1 mn, so except for a couple of mn per day, the M5PaperS3 will be powered down, consuming almost no power.
And, as the M5PaperS3 is fitted with a battery, the system can run many days without being plugged to the wall.
Off course, while being powered down, the display will be on. This is the magic of e-paper.
How to select which newspaper's cover will be displayed the next time the system wakes up ?
This is simple! You just have to change the position of the M5PaperS3 on its stand (thanks to the built-in accelerometer).
Portrait, handle on top: Liberation
Portrait, handle on bottom: The New Yorker
Landscape, handle on the right: The New York Times
Landscape, handle on the left: China Daily
If you want to use the system totally unattended, there is nothing more to it. Just put it somewhere you can glance at it, and position it as per your News preference.
The M5PaperS3 is fitted with a touch display, so let's build some interaction.
One can press one of the corner of the display, to scroll thru the last covers stored on the SD card (I typically keep the last 30 days or so). To each of the four corner corresponds one of the four newspapers.
This is an handy way to visualize what happened in the world, in case you missed it.
And for a bit of fun: shake the M5PaperS3: It will display a cover at random (shaking is detected using the built-in gyroscope).
If you stop messing around with the M5PaperS3 for a while, it will power off, getting ready for the next wake up.
Also, as you as you are tired to interact with the M5PaperS3, just double touch: it will power off immediately.
The pictures are stored on an SD card. I have plenty of old SD card laying around, and any capacity will do. Using SD for storage has also the advantage that the M5PaperS3's internal flash can be erased, without loosing the already stored pictures.
So far I made a passing statement "grab the latest set of covers available online". Is this that simple ?
The New York Times provides every day, on a well-known URL, a PDF with the paper's cover page.
But this PDF needs to be cropped, resized, converted to grayscale and lastly converted to jpg to be nicely displayable on the M5PaperS3's display.
The other newspapers are not that kind.
Typically their cover page is "inside" the newspaper's web site. The cover page is a JPEG file stored "somewhere" but this "somewhere" is not documented, it is just buried in the HTML definition of the web site.
Crap..
Scrap itWeb scrapping comes to the rescue. It is a technique to navigate a web site's HTML tree (jungle ?), to look for information of interest.
It may look intimidating, but using a python library called "beautifulsoup" it is actually quite simple.
First open the site in a browser, position the mouse to the element of interest (i.e. the cover page), right click and select inspect.
The html definition for the cover page (including the sought after JPEG filename) will be displayed on the right. I will look something like:
<img class="ui image" src="//static.milibris.com/thumbnail/issue/a5dfef92-d9d1-41ca-956f-18871b18f442/front/catalog-cover-large.jpeg">Beautifulsoup allows to traverse the tree of HTML tags and once in the appropriate context, look for tag of a specific type (in our case <img>) and then extract the data of interest in that tag (in our case "src") which is the url of the cover's jpeg).
A simple http get on this URL allows to get the cover's JPEG file itself.
It takes a bit of practice and exploration, but it is rather simple and fun.
I used this technique to get covers for China Daily, Liberation and The New Yorker. For China Daily, I also extracted the English caption, which is added to the JPEG itself.
As for the New York Times, the JPEG needs to be cropped, resized and converted to gray scale.
Micro-python and Python togetherThe process described above (scrapping, cropping, resizing, conversion from PDF toJPG, conversion to grayscale) is too heavy duty for micro-python (in particular I did not find a micro-python web scrapping library)
So it is implemented in Python on a Raspberry PI, and exposed via a REST interface
When wanting content from the internet, the M5PaperS3 accesses a well-known URL on the PI. For example:
http://192.168.1.207:5500/china_dailyThe PI will scrap the web, prepare a file ready for display on the M5PaperS3 and return a JSON with a URL on the PI's web server
{"L":"http://192.168.1.207:81/china_daily/0_china_daily_epaper_L.jpg","nb":4,"ok":true,"org":"http://192.168.1.207:81/china_daily/china_daily_org.jpg"}The M5PaperS3 then grab, from the PI's web server, the file ready for e-paper (L here stands for Luminance, i.e. meant for grayscale display)
http://192.168.1.207:81/china_daily/0_china_daily_epaper_L.jpgThe process running on the PI is managed as a daemon by systemd. The python source is available on github. A requirement file is provided.
Comments