Pipeless: An open source computer vision framework

Pipeless is an open-source computer vision framework to create and deploy applications without the complexity of building and maintaining multimedia pipelines. It ships everything you need to create and deploy efficient computer vision applications that work in real-time in just minutes.

Pipeless is inspired by modern serverless technologies. It provides the development experience of serverless frameworks applied to computer vision. You provide some functions that are executed for new video frames and Pipeless takes care of everything else.

You can easily use industry-standard models, such as YOLO, or load your custom model in one of the supported inference runtimes. Pipeless ships some of the most popular inference runtimes, such as the ONNX Runtime, allowing you to run inference with high performance on CPU or GPU out-of-the-box.

You can deploy your Pipeless application to embedded edge and IoT devices or the cloud, all with a single command.

Pipeless, among other features, supports: multi-stream processing, dynamic stream configuration, several inference runtimes (including CUDA, TensorRT, OpenVINO, CoreML), multiple programming languages for your functions ) including Python and Rust), and runs your functions in a highly parallelized manner out-of-the-box. It simply does all the complex stuff for you.

Pipeless was born out of my frustration creating a proxy to censor content on a Twitch live stream. Like many others, I often ended up creating my own custom multimedia pipelines and felt that I was reinventing the wheel and dealing with “plumbing” instead of working on my app.

Even though computer vision is not a new technology, it is ongoing rapid change and you could argue it is still in the early stages of adoption by the wider industry. The same goes for the systems where these applications are deployed. Sending and analyzing video has been quite a resource-intensive task, though recently we are starting to have affordable hardware for it as well as widespread faster internet connections.

This evolution has parallels to how the web evolved. When it appeared in the 1990s, it was primarily used by researchers. Over the years, the addition of CSS, Javascript and other technologies allowed the creation of more dynamic, interactive sites. Later on, the rise of single page applications and frameworks such as Angular and React made it really simple to build powerful web applications to rival its desktop counterparts. We are now in an era of serverless platforms like Vercel and frameworks like Next.js, that make it trivial to build and deploy efficient web applications.

The computer vision field was also started by researchers. Later on, open source models as well as libraries like TensorFlow, PyTorch or OpenCV made it easier to develop applications, and it started to be used in the real world such as in self-driving cars, medical analysis, facial recognition, etc. However, it was still hard to build and create these applications. Now, as the number and sophistication of these apps grows, our goal with Pipeless is to simplify the process of creating and deploying them, significantly lowering the barrier to adoption.

In other words, Pipeless aims to deliver for computer vision what Reactand Next.js brought to the web landscape. They made it simple to get started, build from scratch and deploy an application in just minutes. You don’t need to worry about maintaining servers, you simply implement some functions that get invoked on requests. Pipeless follows a very similar approach. With Pipeless you just implement some hooks (functions) that get invoked when there is a new video frame. You don’t need to worry about servers, you don’t need to worry about video formats or multimedia pipelines, and, in some cases, you don’t even need to worry about bringing your computer vision models, you simply implement some functions and deploy your application, while the rest is automatically handled by Pipeless.

Pipeless will boost the development of new vision applications by a large number of developers for whom the current barrier of entry is too high. This is a step further towards the goal of making computer vision as mainstream as having computers interact with the world via a keyboard or mouse. Our early adopters consistently remark how easy it was to get started and quickly get impressive proof of concepts up and running with just a few lines of Python.

You can find a very quick guide to start using it and several examples in the documentation of the project at https://www.pipeless.ai/docs/docs/v1/getting-started.

Find a demo video installing and running one of the Pipeless examples at https://www.youtube.com/watch?v=4i0tXBhI9zk