1. What problem are you going to solve?
The main problem to be addressed is the technical constraints of video recordings, which often lacks the quality needed for detailed viewing. This is particularly challenging for creating high-quality video content for marketing, promotional, or other purposes. The goal is to enhance the quality of video recordings using AI-based upscaling techniques, specifically targeting the optimization of these techniques for accelerated inference on AMD Ryzen AI hardware.
2. What are you going to build to solve this problem? How is it different from existing solutions? Why is it useful?
Proposed Solution
Accelerating the Stable Diffusion image enhancement model (Upscaler) to upscale video files, leveraging AMD Ryzen AI for optimized performance. This approach will enable the processing of various video recordings with high efficiency and quality, making it suitable for creating detailed and visually appealing video content.
Improvements from Existing SolutionsAI Based Accelerated High-Quality Upscaling: Unlike generic video upscaling solutions, this AI-based and hardware acceleration approach specifically tailors the upscaling process to enhance visual quality significantly.
Optimized for AMD Ryzen AI: The solution leverages AMD's AI hardware to accelerate the inference process, ensuring faster and more efficient processing compared to standard CPU or GPU methods.
Hardware: AMD Ryzen AI Powered PC
Software:
1. AMD Ryzen AI SW
2. Stable Diffusion Upscaler model (PyTorch)
3. Python, ONNX, Pytorch
4. Vitis AI quantizer
5. FFMPEG for video frame extraction and reconstruction
3. How does your solution work? What are the main features? Please specify how you will use the AMD AI Hardware in your solution.
- Post and PreProcessing for Frame Extraction and Reconstruction: Utilize FFMPEG to extract frames from videos, upscale them using the Stable Diffusion model, and recombine them to maintain the same FPS as the input video.
- Model Replication and Conversion: Replicate the Stable Diffusion UpScaler model and convert it to ONNX format for compatibility with AMD Ryzen AI SW.
- Model Optimization: Use Vitis AI quantizer to optimize the ONNX model for accelerated inference using the ONNX Runtime execution provider. Source
- Inference API: Develop a Python API for efficient inference on AMD Ryzen AI hardware, enabling seamless integration into existing workflows.
Did not take the hardware quiz
References
Comments