The Fire Pot: reimagining hotpot

1 Introduction

1.1 The problem: guessing at the hotpot stand

There's a hotpot stall in a Singapore hawker centre that I ate at all the way through my master's at NTU. The arrangement is beautifully simple. Take a bowl, take a pair of tongs, walk along the stand and load up whatever you fancy. Hand it over at the end. They weigh it, ask which broth and how spicy, and cook it. Your number gets called. It's fast, it's cheap, and you get exactly the bowl you asked for.

I ate there constantly, and I was guessing every single time.

Some bins I could read on sight. Others held a pale, smooth sphere that could have been fish, could have been pork, could have been a very confident potato. So I picked by shape. I learned what any of it tasted like only once it came back cooked, in a broth I'd chosen off a sign written in a language I can't read, at a price I found out at the till.

Everyone else seemed to know. The staff knew. The regulars knew. The answers existed. They were just somewhere other than where I was standing, which was at the stand, holding tongs, holding up the queue.

The Fire Pot puts the answers exactly where I was standing.

1.2 The idea: put the answers on the table

Everything I wanted to know was a signage problem. What is this, what does it taste like, what will it cost me — none of it is hard to know, it just has nowhere to go. A printed card on each bin goes stale by lunchtime and still can't tell you what's in your bowl. A menu board hangs the answer behind the counter, when the question is in front of you, in your hands. An app asks someone holding tongs in one hand and a bowl in the other to hold a phone as well.

So the sign comes down off the wall and lands on the table.

The Fire Pot is a self-serve hotpot counter with eight bins set into it. A projector overhead turns the whole tabletop into a display, a camera beside it watches your hands, and there is a scale under every bin. The signage is projected straight onto the food it describes, and it answers back: reach for a bin and it tells you what is in it, scoop it into your bowl and it prices what you took. Nothing to tap, nothing to hold, nothing to ask anyone.

2 Using the table

2.1 Idle state: breathing halos and a wandering flame

Every bin wears a soft golden ring that breathes, brightening and fading slowly, each one a little out of step with its neighbours, so the surface is never entirely still.

What gives away an empty table is the fireball. Left to itself, a small flame wanders the surface, drifting from bin to bin at browsing pace, as though someone invisible were working along the row and taking their time about it. Underneath, the low crackle of a fire.

Walk up, put a hand over the table, and it comes to attention. All eight bins label themselves at once: the name of what's in them, and the price per 100 g underneath.

Instant Noodles, $1.50. Lotus Root Slices, $1.40. Dried Eel Strips, $5.00. You know what everything is and exactly what it costs before you pick up a single thing.

2.2 Selecting a bin: the flame follows your hand

Put a hand over the table and the wandering flame becomes yours. The fireball leaves off drifting and tracks your palm instead, staying with you wherever you move.

Carry it over a bin and the fire changes hands. The ball goes out, the bin's golden ring gives way to a burning one, and it is the bin that's alight now, in a colour of its own. Lift your hand away from the bin and the flame comes back to you, the bin settling behind you into its slow gold breathing again.

While a bin is burning, the middle of the table fills with the thing you're pointing at:

• What it is: the full name, in large type
• Veg, non-veg, or egg: a coloured dot with the word beside it
• Calories per 100 g: Button Mushrooms come in at 22, Instant Noodles at 450, which is the single most useful number nobody ever tells you
• What it actually tastes like: one honest sentence, written for someone who has never eaten it

That last one is the whole reason this project exists.

• Lotus Root Slices: stays crunchy even after cooking. Mild, faintly sweet, never goes soft.
• Dried Shrimp: small but intense. Deeply savoury, so a little goes a long way.
• Fish Balls: bouncy and mild. A safe pick if you would rather not chase strong flavours.

Move your hand along the row and you can read your way through all eight before you commit to one.

2.3 The cart: live weight and running total

Scoop something into your bowl and it appears in the cart along the near edge of the table, with its weight and its price. Scoop more and the number climbs. Every item you've picked stays listed, and the running total sits underneath the lot, ticking upward as it goes.

This is the number I never had in Singapore. It means you can take 60 g of the $5.00 eel because you can see it's costing you $3.00, and you can be generous with the $1.00 potato because you can see that it barely registers. The expensive decisions and the cheap ones stop feeling the same.

Take too much? Tip some back into the bin. The weight drops, the price drops with it, and the total settles to the new figure. The table simply reports what's in your bowl at this moment, so you can rearrange your order as many times as you like right up until you're happy with it.

2.4 Switching language: English and Chinese

One button, marked EN | 中文, switches the entire table over. Every ingredient name, every tasting note, every price, the broth descriptions, the spice levels, the checkout. All of it.

鱼丸. 藕片. 牛肉丸.

It switches back just as fast, so two people reading different languages can share the same table and take turns.

2.5 Choosing a broth

When you're done picking, the table turns into the ordering screen, and the first question is the broth. Three of them, each presented with the two facts that actually decide it:

• Classic Mala Broth.Very spicy. Numbing and fiery, built on Sichuan pepper and chilli. The boldest broth here.
• Mushroom Vegan Broth.Not spicy. Deep and savoury with no meat at all. Earthy rather than rich.
• Collagen Bone Broth.Not spicy. Milky and slow simmered. Rich on the tongue but completely mild.

Each one says whether it's spicy and whether it's vegetarian right on the card, in your language, before you commit. This is the sign I stood in front of in Singapore and couldn't read, rewritten so that a first-timer can pick with confidence.

2.6 Choosing a spice level

Spice arrives as the same three cards, stacked by heat. Hot sits at the top of the table, furthest from you; mild at the bottom, nearest to hand. Each card counts its own heat in chillies: one, two, three.

• Hot. Serious heat. Worth choosing only if you already know you like it.
• Medium. Properly spicy and still comfortable. The most ordered level.
• Mild. A gentle warmth. You taste the broth first and the chilli after.

Pick one and the table moves on to the total. A row of chillies means the same thing in every language, which is rather the point.

2.7 Paying and collecting a token number

The table adds up your ingredients, your broth and your spice, shows you the total, and projects a QR code onto the table in front of you.

Scan it with your phone and you get an itemised receipt: every ingredient with its weight and its price, the broth you picked, the spice level you picked, and the total at the bottom. Pay on the phone, and the table notices immediately.

It clears down to a single large token number, with two lines under it: hand your bowl to the staff, and we'll call this number when it's ready. The staff take the bowl away and cook it. Your number gets called. Exactly the same ending as the stall in Singapore. The difference is everything that happened before it.

2.8 Dwell-to-press, with sound demo

3 The staff dashboard

Everything above is what a diner sees. The staff run the table from a browser on any device on the same network, and it's where the fiddly parts of a real restaurant shift live.

3.1 The Serving switch

One switch at the top says Serving. On, and the table is open for business. Off, and it stops billing so staff can reach across the bins, swap trays and work on the table without anything registering as a customer's pick.

3.2 Bins tab: weights, stock alerts and scale calibration

Eight cards, one per bin, each showing its live weight as it changes. A dropdown on every card says what that bin currently holds, so refilling bin 3 with something different is a two-second job. Let a bin fall below 50 g and its card raises a restock alert, which saves anyone the walk to go and look. The scales themselves live here too: an empty bin can be zeroed, and any bin can be calibrated against a known weight. Zeroing runs on all eight at once, since "the bins are empty" tends to be true of the lot of them at the same moment.

3.3 Setup tab: table corners and bin grids

This is the tab that teaches the table where it is. The staff mark the four corners of the physical table on a live camera view, then lay a grid over it to say where each bin sits, and finally nudge a second grid with the arrow keys, watching the real projected light on the real table, until it lines up with the actual bins. It's the one job that has to be done by a human looking at the thing itself, and it's built to be done that way.

3.4 Capture tab: building a training dataset

The table can photograph its own bins to build a training dataset, one burst at a time, at whatever interval suits the person rearranging the tray between shots. Those images export straight out to Edge Impulse or Roboflow.

3.5 Developer tab: hand tracking, camera and load cell plots

The engineering window. A live hand skeleton overlaid on the camera feed showing exactly what the tracker sees, the classifier's (Classifier AI Edge impulse project link) guesses as they happen, camera controls for exposure and white balance, a health readout for every process, and a live plot of each load cell, raw against filtered, with the filter's own settings adjustable while you watch the line change shape.

4 System overview

4.1 Hardware at a glance

The tabletop rests on eight load cells, one under each bin, read by a Seeed Studio XIAO ESP32S3 that streams all eight channels up to the host over USB. The host is an ASUS NUC 14, running the brains in Python and the projected table itself in openFrameworks — I had wanted a Seeed Studio reComputer for that job and simply could not get hold of one, so the NUC took its place. Above it all, an ordinary projector mounted almost against the ceiling, high enough that its throw covers the full table, with a webcam beside it and Google's MediaPipe doing the hand tracking.

Everything past this point is the engineering. If you want to build one, follow sections 5, 6 and 7. If you want to understand how it works, section 8 covers the architecture.

5 Building the hardware

5.1 Bill of materials

5.1.1 Sensing and wiring

• 1 x Seeed Studio XIAO ESP32S3
• 8 x Phaeton CZL-611N 1 kg screw-fastened load cell, one per bin
• 8 x SmartElex HX711 load cell amplifier, one per cell
• 16 x M4 bolts, 15 mm, and 16 x M5 bolts, 15 mm. Two of each per load cell
• 1 x 5 x 7 cm universal prototype board, 2.54 mm pitch, the carrier the XIAO sits on
• 1 x 2.54 mm 1x40 female single row header strip
• 12 x 4-pin 2.54 mm pluggable screw terminal blocks: one per HX711, and four on the XIAO board
• 6 x PCT-215 5-pole lever connectors, to split 3V3, ground and the clock four ways
• 26 AWG ultra flexible silicone wire in red, black, blue, yellow, green and white

5.1.2 Light, camera and computer

• 1 x PC: the host machine
• 1 x WZATCO Yuva Go Plus projector, 1080p native, 9000 lumens, with 4D keystone on the projector itself
• 1 x Lenovo 300 FHD webcam, 1080p, 95 degree lens, on a flexible mount
• 1 x HDMI cable long enough to reach the ceiling
• 2 x USB extension cable, one for the camera and one for the XIAO

5.1.3 The table

• A table with a top of roughly 1524 x 914.4 mm, which carries the bins and everything under them
• 6 mm plywood, cut to 1524 x 914.4 mm, which becomes the projection surface above it
• Projector screen fabric to cover the plywood
• 8 x bins
• 1 kg of eSUN PLA, 1.75 mm, gold for the mounts and the pillars
• A little eSUN PLA, 1.75 mm, white for the corner plugs

5.2 Printing the parts

Every printed part is in hardware/3d-printed/ in the repo, as .3mf plates ready to slice. All PLA.

In gold:

• load_base.3mf, 145 x 160 x 10 mm, a cross that sits on the table. Print 8, one per bin.
• load_plate_short.3mf, 200 x 25 x 35 mm. Print 16, two per bin.
• load_plate_long.3mf, 25 x 254 x 35 mm. Print 16, two per bin.
• pillar.3mf, a 40 x 40 x 70 mm tapered leg. Print 12. These hold the plywood up.

In white:

• corner_plug.3mf, 25 x 25 x 7 mm. Print 32, four per bin opening. These round off the corners of the openings once the plywood is covered.

Two short rails and two long rails make one bin's platform, 200 x 255 mm, which is the footprint the whole system is built around. The load cell bolts between that platform and the cross base with two M4 and two M5 bolts.

5.3 Wiring the load cells

5.3.1 Assembling the harness

Every connection is a screw terminal or a lever connector, so the whole harness comes apart again.

• Solder the female header strip to the 5 x 7 cm prototype board and plug the XIAO into it
• Put four screw terminal blocks around the XIAO: one power and clock block and one data block per island
• Give every HX711 its own screw terminal block for the run back to the XIAO
• Split 3V3, ground and the clock with the PCT-215 lever connectors, one feed in and four out, three per island

5.3.2 Load cell to HX711 connections

Short VCC and VDD together on each HX711. Both go to 3V3.

Each load cell has four wires into its own HX711: red to E+, black to E-, green to A+, white to A-. Check the colours against your own cell.

5.3.3 XIAO pin map, bin by bin

Each HX711 then takes 3V3 and ground from the rails, shares the clock on D0 with all seven others, and sends its own data line back:

D3  / GPIO4     bin 0    far left
D4  / GPIO5     bin 1    far centre-left
D10 / GPIO9     bin 2    far centre-right
D9  / GPIO8     bin 3    far right
D5  / GPIO6     bin 4    near left
D6  / GPIO43    bin 5    near centre-left
D8  / GPIO7     bin 6    near centre-right
D7  / GPIO44    bin 7    near right

The eight bins sit as two islands of four, each island holding one pair from each row. D3 to D6 run to the left island and D7 to D10 to the right, and the right island counts backwards because D10 down to D7 sit top to bottom on that header. Wire it in that order and the harness stays in physical order.

Leave D1 and D2 unused. D2 is GPIO3, a strapping pin sampled at reset.

Mount each cell whichever way round suits the frame. Calibration works out the sign per bin.

5.4 Cutting the plywood

The plywood is the projection surface. It is 6 mm thick and carries no weight: it stands on the twelve printed pillars at the height of the bin tops, and each bin comes up through its own opening.

Cut the sheet to 1524 x 914.4 mm, which is 60 x 36 inches, then cut the eight openings at the positions in the plan. Both chains have to add up:

across:  92 + 200 + 50 + 200 + 440 + 200 + 50 + 200 + 92 = 1524 mm
front to back:     177 + 255 +  50 + 255 + 177.4         = 914.4 mm

The 440 mm column up the middle stays solid. That is where the table projects the ingredient details, the cart and the checkout buttons.

• Cut every opening at 200 x 255 mm, taking the four corners off at an angle
• Lay the screen fabric over the plywood and cut the openings through it
• Glue the fabric down
• Glue a white corner plug into each corner of each opening, which is what makes the corners round

Stand the plywood on the pillars over the table, with each bin coming up through its opening.

5.5 Hanging the projector and the camera

Mount the projector on the wall, right against the ceiling, aimed straight down at the table. Set it back far enough that the image covers the full 1524 mm width. Square the image with the projector's own 4D keystone.

The camera rides on the projector. A wooden batten is fixed to the projector's mount and sticks out from it. Zip tie the camera to the end of that batten, looking down at the same table. On this build the camera hangs upside down — 180 degrees.

Cabling:

• HDMI to the projector
• One USB extension to the camera
• One USB extension to the XIAO
• The camera needs a USB 3 port on the host; the XIAO doesn't

6 Installing the software

6.1 Getting the code

The projected table and the Python processes behind it live in one repository, and it belongs inside an openFrameworks tree: the Visual Studio project walks five levels up to find openFrameworks, so apps/myApps/hotpot-table is where it expects to be.

Download openFrameworks 0.12.1 for Visual Studio, unzip it, and clone into it:

cd openFrameworks/apps/myApps
git clone https://github.com/rahuljeyaraj/hotpot-table

Everything from here on runs from that folder.

6.2 Flashing the XIAO

Flash the firmware with PlatformIO:

cd openFrameworks/apps/myApps/hotpot-table/firmware/loadcells
pio run -t upload
pio device monitor

A working board prints this and then keeps going:

# hx711 loadcell aggregator, 8 cells, raw counts
# built Aug 28 2026 11:04:12
# probe: all 8 cells reporting
# ready
raw 83422 -211904 84001 82755 -210338 83190 84772 83006

6.2.1 Reading the probe output

If a cell is missing, the probe names the pin and says which of two faults it is:

# probe: no data line activity from bins 2 5 (D10/GPIO9 D6/GPIO43)
#   bin 2 D10/GPIO9 floating
#   bin 5 D6/GPIO43 driven-high

• floating: an open data wire, or an HX711 with no power. Check that terminal block.
• driven-high: the board has power and is holding the line up. Check the clock reaching it, and the board itself.

6.3 Installing dependencies

6.3.1 openFrameworks addons and the ofxFlowTools patch

Four addons are needed. ofxGui, ofxOsc and ofxNetwork come with openFrameworks. ofxFlowTools, the fluid solver behind the fire, is a separate download that goes in openFrameworks/addons/.

It needs a patch before it will build, kept in the repo at of/patches/ofxFlowTools.patch:

cd openFrameworks/addons/ofxFlowTools
git apply ../../apps/myApps/hotpot-table/of/patches/ofxFlowTools.patch

That does two things. It replaces two constructs the current compiler rejects, std::bind2nd and an unqualified min. And it fixes the density lookup in the buoyancy and diffusion shaders, which sample a texture that is at a different resolution from the one they render at, so the fire is pulled around by density that sits at double the coordinates of the real thing. There is more on that one in the next section.

Open of/hotpot-table/hotpot-table.sln in Visual Studio and build the Debug configuration, which is the binary the launcher starts. Set HOTPOT_OF_BIN if you would rather it ran something else.

6.3.2 Python environment

One interpreter for the whole tree, installed from the repository root:

pip install -r python/requirements.txt

6.3.3 Three values for your own rig

• SCALE_PORT in python/hotpot/core/main.py is the XIAO's serial port, COM5 here. pio device monitor names yours.
• camera.windows_device_index in config/system.json picks the webcam, 0 by default. Raise it if the host has another camera it prefers.
• of/hotpot-table/bin/data/display.txt picks the monitor, as the projector's desktop origin in x,y form. Start the app once and it writes the file and logs every monitor it found with its origin. Put the projector's in. This rig's reads -1920,144.

6.4 Starting the system

python run.py

That starts and supervises everything, waits for each process to report ready before starting the next, and writes one merged log to the terminal and to logs/hotpot-<date>.log.

• python run.py --replace stops whatever is running and starts fresh
• python run.py --stop stops a detached instance
• python run.py --only core,of starts the table alone, with the camera unplugged
• python run.py --no-restart leaves a crashed process dead so you can read its traceback

The dashboard is at http://<the-host>:8090 from any device on the same network, a phone included.

7 Calibration and first run

7.1 Calibrating the geometry

Three steps in the dashboard's Setup tab, in the order the cards appear. Turn the Serving switch off first: every one of them is refused while it is on.

7.1.1 Marking the four table corners

Tap Set table corners. A quad appears on the live camera feed with a handle at each corner. Drag each handle onto the matching real corner of the table. The handles have fixed roles, near-left, near-right, far-right, far-left, so read the real table rather than the screen. Holding a handle brings up a 4x magnifier. Tap Confirm, and both the Setup and Live tabs switch to a flattened, top-down view of the table.

7.1.2 Laying the bin grid

Four horizontal lines and eight vertical ones, dragged onto the flattened view: a horizontal line sets a whole row's edge, a vertical line sets a whole column's edge. Load measured layout starts you from the CAD positions, Undo steps back, and Save commits it.

7.1.3 Nudging the projector grid

Click one of the twelve lines in the list, or Tab to it, and press the arrow keys. Up and down move a horizontal line, left and right move a vertical one, one pixel per press and ten with Shift held. Each press reaches the projector immediately and there is nothing to save. Work through the lines until each projected golden halo sits on its real bin, watching the table rather than the screen.

7.2 Calibrating the scales

Serving stays off for this too.

Go to the Bins tab, empty all eight bins, and tap Tare all 8 bins. It zeroes every bin from one 2-second reading.

Then, per bin, tap Calibrate, put a known weight in that bin, type its mass and confirm. A result under ten counts per gram is refused, which means the cell has come loose or the weight was too light.

Finally, set what each bin holds from the dropdown on its card.

7.3 Checking it works

Turn Serving on and stand at the table.

• Eight golden halos, breathing, and with nobody at the table a flame wandering from bin to bin
• A hand over the table brings up all eight names and prices
• A hand over one bin moves the flame onto it and fills the middle column with that ingredient's details
• Something taken from a bin appears in the cart with a weight that matches your kitchen scales
• Putting it back takes it off again

To check the table before the load cells are wired, start python run.py --only core,of and open the Developer tab. It has a Pick and a Put back button per bin, and they drive the whole flow through to the token number.

8 How it works

The table looks like one thing: a lit surface that reacts to your hands. It is actually five separate programs running at once, talking to each other over the network and over shared memory. This section is a tour of how they are split up and why.

8.1 Five processes

Five programs run at once, and not one of them shares a variable with any other. Everything they know about each other comes down the arrows in that diagram.

• camera owns the webcam. It sets exposure and white balance once, locks them so the picture stops drifting, and publishes every frame into shared memory.
• tracker reads those frames, finds the hand in them, and sends the fingertip out sixty times a second.
• classifier reads the same frames, crops out each bin, and works out what is in it.
• core is everything the table means: the state machine, the cart, the prices, the bin map, the geometry, the load cell readings coming in over USB serial, the order database and the staff dashboard. Every other process dials into it.
• of is the openFrameworks renderer. It draws the table exactly as core describes it, sixty times a second.

python run.py is the whole start command. It launches all five, waits for each to announce itself, merges every log into one stream, and restarts anything that dies. The order it starts them in is only a convenience: every process reconnects forever, so any one of them can die and come back at any moment and the table carries on.

8.2 Three transports for three kinds of traffic

Three kinds of traffic pass between the processes, and each one gets the transport that suits it rather than a single channel carrying all three.

• Where your hand is — UDP, port 8770. Sixty datagrams a second from the tracker to the renderer. A lost cursor packet is worthless sixteen milliseconds later, and a queued one is worse: when the renderer catches up it delivers the whole backlog in order and the hand visibly walks back through its own history. So the rule at the far end is drain the socket, keep the newest, throw the rest away.
• What the camera sees — shared memory. Frames are six megabytes each, thirty times a second, far too much to push down a socket. They go in a ring of eight slots, about 47 MB, one writer and two readers with no lock. core never imports that module at all, and the omission is the enforcement: the process that handles money has no way to touch a pixel.
• Everything else — TCP, port 8765. The state machine, the cart, the prices, the bin map, the geometry, the load cell readings. core is the server and every other process dials into it, so the whole table's truth arrives down one connection, sixty times a second, already translated into whichever language is selected.

8.3 One coordinate system on the wire

Every position sent between the processes is in projector pixels, 1920 by 1080. Never millimetres, never camera pixels, so no two processes can disagree about where a bin is.

Two things convert into it: the table, measured in millimetres, and the camera, which someone lines up by dragging four corners onto the live video. Nothing checks that alignment for you — the test is whether the light lands on the real trays.

The bin boundaries are stored twice, once as the camera sees them and once as the projector draws them, because a rectangle that looks right in one view can still be wrong in the other.

8.4 The eight states

Everything the table does is one of eight states, and core is the only process that decides which one it is in.

• BOOT. Start-up. It goes straight to IDLE if there is saved geometry on disk, and to UNCALIBRATED if there is not.
• UNCALIBRATED. No geometry, so the table has no idea which tray is which. Serving cannot be turned on from here.
• IDLE. Nobody at the table. Breathing halos, a flame wandering from bin to bin.
• SELECTING. A hand has arrived. Bins label themselves, the flame follows the hand, and the cart fills as food is taken.
• BROTH. Choose a broth.
• SPICE. Choose a spice level.
• CHECKOUT. The total, then payment, then a token number, then back to IDLE.
• SETTING. Serving switched off: the staff mode for swapping a tray, zeroing a scale or redoing the geometry. It can be entered from any of the other seven.

The four middle screens each have a Back button, and Cancel returns to IDLE from any of them.

8.5 The bill is a subtraction

The cart keeps two numbers per bin: what it weighed when you walked up, and what it weighs now. Every price on the table is the difference between them, worked out afresh sixty times a second, and nothing anywhere adds up individual scoops. That is also why putting food back works without any code for it — the same subtraction, a smaller answer.

8.6 The projector lights the food it is labelling

A frame is built up in five layers, bottom to top:

• The table background. A warm near-white, #E8E6E1, cleared first.
• The fire. The fluid simulation, drawn in multiply so it darkens the background instead of brightening it.
• The light pass. Flat opaque white filling each of the eight bin cutouts.
• The halos. A golden ring around each bin.
• The UI. Names, prices, the info box, the cart, the buttons

The two rows of four bins leave a 440 mm strip down the middle of the table, 554 pixels of it, and everything you read is in there. Top to bottom: the brand mark, then the info box for whichever bin your hand is over, then the cart, eight rows and a total, then three buttons along the bottom edge. When Serving is switched off, a mode banner takes the info box's place.

Every one of those bands is positioned from the bin geometry rather than typed in, so moving a bin in the layout moves the reading column with it.

8.7 Hand tracking through a moving window

The camera sees the whole table, but MediaPipe is never shown the whole table. It only ever gets a 700 pixel square, and it is that square which moves.

With nobody at the table the square is a scanner, stepping through overlapping tiles laid over the table's footprint, one tile per tick, each denoised before it is handed over. As soon as a hand turns up in one, the square commits to it and re-centres on the hand every tick from then on, so it travels with the hand.

What comes out is a single point, the index fingertip, put through the camera homography into projector pixels and smoothed so per-frame jitter never reaches the cursor. The table follows one hand at a time.

9 Future improvements

• Automatic camera-to-projector mapping. Drop the four hand-dragged corner handles and let the two devices align themselves.
• The camera finds the table and the bins by itself. No manual grid, no measured layout file - point it at the table and it works out where every bin is.
• More languages. Two is enough to prove the idea; a real restaurant floor needs more.
• Four-point load cells for the larger bins. A single cell under a big tray reads unevenly; four per bin would weigh the bigger ingredients honestly.
• A low stock alert scaled to the bin. The restock alert trips at a flat 50 g in every bin, which is a nearly empty tray of cabbage and most of a small bin of goji berries. It should trip at a fraction of what that bin weighed when it was filled - say 10% - so every bin warns at the same point in its own life rather than at the same number of grams.
• Watching where food goes back. Putting food back already works, because the bill is a subtraction and nothing ever tracked individual scoops. But nothing checks that the lotus root went back into the lotus root bin. The camera is already following the hand and the classifier already knows what every bin holds, so between them they could tell the counter that a bin has been contaminated, and with what.
• A synthesised flame sound. Every other effect is a recorded clip, but the fire changes shape every frame - its sound should be generated from the sim, not played back.
• Support more than one camera-projector pair. Re-architect so several pairs can run side by side and cover a longer table.

10 Conclusion

The table is built and it works. Eight bins set into a sheet of plywood, a projector on a stand above it, one webcam, and a load cell under every tray.

Walk up to it and the food labels itself. Each bin says what it holds and what it costs per 100 g, projected onto the food rather than printed on a card or hung on a board behind the counter. Reach for a bin and it tells you more about that ingredient. Scoop some into your bowl and the price appears in a cart on the table, because a bin that weighs less than it did a minute ago is food you took. Nothing to tap, nothing to hold, and nobody to ask.

I picked by shape at that stand for a whole master's degree. On this table I would have known what was in the bowl, what it cost and how it would taste before I picked up a single piece — and so would the person behind me in the queue, in whichever of the two languages they read.