Home Exam 1: Video Encoding on ARM Cortex-A57 using ARM NEON Instructions
Assignment
In this assignment, we will take advantage of the parallelization options available in a single ARMv8 NEON-enabled core to accelerate Codec63 (c63) video encoding.
You are supposed to:
- Optimize the c63 encoder using ARM NEON vector instructions.
- Write a short report where you describe which optimizations you have implemented and discuss your results. You should not describe other thinkable or planned optimizations that you did not test.
- Create a poster (2 x A3 pages) that the group will present on Tuesday September 26th.
Additional details
The exam will be graded on how well you are able to take advantage of the ARM NEON instructions to solve the task.
You are not supposed to make the encoder multi-threaded. Your implementation is supposed to be single threaded, and optimized to use the parallelism available through NEON vector instructions.
The encoder must accept the test sequences in YUV format and generate the format that is understood by c63's own and unmodified decoder.
Start by profiling the encoder to see which parts of the encoder that are the bottlenecks. Remember, after optimizing one part of the code, more profiling might be needed to find new bottlenecks. One operation (e.g., motion vector search) may still be the heaviest function when you have optimised as much as you can. If you cannot optimize one operation further, move to another one.
Based on your profiling you should try to optimize different parts of the code, (1) structurally, and (2) with NEON instructions. There is no definite answer to which parts of the code you have to optimize, and there is also no definite answer to which instructions you have to use. Look for SIMD-friendly cases where the same operation needs to be performed on very many similar data elements. You are NOT supposed to change or replace any algorithms, only reimplement the algorithms using vector instructions.
Write a report detailing your profiling results, the instructions you used, and your changes to the precode. The report should also detail and explain the performance results, both positive and negative (in research it is also important to learn what not to do). If you find several alternatives for solving a problem, or if you tried several dead ends before solving a problem successfully, discuss them in your report.
Codec63
Codec63 is a modified variant of Motion JPEG that supports inter-frame prediction. It provides a variety of parallelization opportunities that exist in modern codecs without the complexity of those full-fledged codecs. It is not compliant with any standards by itself, so the precode contains both an example of an encoder and a decoder (which converts an encoded file back to YUV).
C63's inter-frame prediction works by encoding every macroblock independently whether it uses a motion vector or not. If a motion vector is used, it refers to the previous frame. If a motion vector is used, the residual is stored in the same manner. If a motion vector is used, this is stored right before storing the encoded residual.
If no motion vector is used, the macroblocks are encoded according to the JPEG standard [1] and stored in the output file. An illustrative overview of the steps involved during JPEG encoding can be found at Wikipedia [2].
The c63 is very basic and shows behavior that you would not allow a standard encoder to have. This concerns in particular the Huffman tables and the unconditional use of motion vectors in non-I-frames. You should not modify these Huffman tables. You can decide to use conditional motion vectors, but you must search for motion vectors, and you must write code that potentially uses the whole motion vector search range (hard-coded to 16 in the precode).
The video scenario is live streaming. You should not use parallelization techniques that severely degrade the video quality.
It is your task to optimize the c63 encoder. As mentioned above, you should not replace the algorithms that you find in c63. Alternative motion vector search algorithms and DCT encoding algorithms may provide large speedups, but they distract from the main goal of this home exam, which is to identify and implement parallelization using vector instructions.
Two test sequences in YUV format is available in the /mnt/sdcard directory on the Tegra-machines:
- foreman (352x288) CIF
- tractor (1920x1080) 1080p
These should be used as input to the provided c63 encoder, and can be used to test your implementations.
Precode
The precode consists of:
- The reference c63 encoder (c63enc).
- The decoder for c63 (c63dec).
- The command c63pred (which extracts the prediction buffer for debugging purposes).
The precode is written in C, and you should also write your solution in C. You are not required to touch the decoder (c63dec) or c63pred. We recommend that you have two separate repositories. One where you modify the encoder, and one unmodified which you use to test your implementation.
The precode can be downloaded from a Git repository here:
git clone https://bitbucket.org/mpg_code/inf5063-codec63.git
You must login to the Jetson TX1 devkit assigned to your group for this assignment. You should have received an email from the course administrators about which kits to use. Information about how to access the kits can be found in the ARM FAQ.
You are free to adapt, modify or completely rewrite the provided encoder to take full advantage of the target architecture. You are, however, not allowed to replace the algorithms for Motion Estimation, Motion Compensation or DCT/iDCT. You are also not allowed to paste/reuse any other pre-written code in your implementation.
Some command usage examples:
To encode the foreman test sequence
$ ./c63enc -w 352 -h 288 -o /tmp/test.c63 foreman.yuv
To decode the test sequence
$ ./c63dec /tmp/test.c63 /tmp/test.yuv
To dump the prediction buffer (used to test motion estimation)
$ ./c63pred /tmp/test.c63 /tmp/test.yuv
$ vlc --rawvid-width 352 --rawvid-height 288 /tmp/test.yuv
Report
You must write up the results as a technical report of no more than 4 pages in ACM format. The report should serve as a guide to the code modifications you have made and the resulting performance changes.
Evaluation
In evaluation, we will consider and give points for (most important first):
- Motion Estimation & DCT/iDCT algorithmic functions in the source code have been NEONized.
- Document the bottleneck and the effect of your optimization.
- A program that works (on the Jetson TX1 provided)
- Runs to completion. (*)
- Encodes foreman (CIF) correctly.
- Output video has a quality with a similar PSNR and file size as the unmodified encoder.
- Readable, well-commented code
- Effect of the Parallelization (SIMDification)
- Most costly algorithms identified and NEONized
- NEON instructions are used effectively
- Bonus points for non-obvious optimizations
- The quality of the report that accompanies the code
- Clear and structured report of the performance changes caused by your modifications to the precode
- References to the relevant parts of the accompanying code (to aid the reviewer of the submitted assignment)
- Graphical presentation of the optimization steps and performance results (plots of performance changes)
- Comparison of / reflection about the alternative approaches tried out by your group
(*) We do not debug code before testing. There will not be any points for correctness if this is not fulfilled.
Machine Setup
The Jetson TX1 devkits are situated at Simula Research Laboratory. Machine names and how to access them can be found in the ARM FAQ. If you have reported your group to the course administration, you should have been assigned to a devkit and provided with a username and a password.
Contact inf5063@ifi.uio.no if you have problems logging in.
Formal information
The deadline for handing in your assignment is: Friday, September 22nd at (16:00:00.00).
Deliver your code and report (as PDF) at https://devilry.ifi.uio.no/. Submit the poster (as PDF) to inf5063@ifi.uio.no.
The groups should prepare a poster (2 x A3 pages) and a quick 2 minutes talk (without slides) where you pitch your poster for the class on September 26th. Name the poster with your group name, and e-mail the poster to inf5063@ifi.uio.no no later than noon (12:00) on September 25th. We will then print the poster for you.
For questions and course related chatter, we have created a Slack space: https://inf5063.slack.com
There will be a prize for best poster/presentation (awarded by an independent panel and independent of the grade).
Please check the ARM FAQ page for updates and FAQ
For questions please contact:
inf5063@ifi.uio.no