Monitoring and control of edge legacy systems such as computers which only provide video/keyboard/mouse input, and other devices (such as fans and air conditioning units) which provide some kind of visual display and are controlled with an IR remote.Existing systems for automating legacy systems (called "Robotic Process Automation" systems) are used for automating web based (non-API) applications, and mainframe style text terminals. Older systems which do not fit the web page or serial terminal model cannot be automated with existing solutions. My proposal is to build a system which will apply similar automation functionality to edge systems that do not have standard network connectivity using computer vision for text and image recognition for input; and keyboard/mouse emulation and IR remote control emulation for output. In addition by performing the video processing at the edge, it allow for usage in low bandwidth or unreliable internet connections.My solution will uses the Xilinx Kria KV260 to perform image recognition from an attached video camera or video input device. The information received can be parsed and logged for monitoring, and trigger a script which can perform actions based upon the input data.In addition to the Xilinx Kira KV260, the a USB Webcam was used.This project is derived from the Vitis AI MNIST sample.
Comments