FFmpeg 7.1
FFmpeg 7.1 has been released today: a major release with numerous features that nevertheless maintains API compatibility with 7.0.
It has a full native VVC decoder, a new MV-HEVC decoder, a new LC-EVC decoder, a new xHE-AAC decoder, it finishes the IAMF decoder and it also adds Vulkan hardware encoding, VVC encoding, ARM64 and RISC-V optimizations and other hardware accelerations.
Knowing that FFmpeg 7.1 is a LTS release, this is a major feat, in addition to the already very strong FFmpeg 7.0 release.
VVC decoder
We decided, a couple of years ago, to do our own VVC decoder, instead of using the official vvcdec decoder.
There are a tons of reasons to prefer native decoders inside FFmpeg, notably for better control of the codebase, but mostly because it allows sharing code between decoders, and this is especially true for the VVC decoder (and other MPEG decoders)
The VVC decoder was a large project, notably done by Google Summer of Code students last year, and the main decoder landed in FFmpeg 7.0, but it was not complete, and was therefore marked as experimental.
The continuation of the work, notably numerous extensions but also optimizations were implemented, which makes that now the decoder can decode DVB streams. This means that the decoder is now production ready in 7.1!. Congratulations to the amazing VVC team, and mostly Nuo Mi and Frank Plowman.
This decoder shares some code with the H.264 and the HEVC decoders, which means that some assembly optimizations are shared or that most SEI parsing is shared (for example, the H.274 Film Grain should work with VVC).
Finally, the VVC decoder is already fast on ARM and on x86, with new optimizations written, and the reuse of the optimizations from the previous decoders!
More VVC
In addition to the decoder, the vvcenc encoder is now available in FFmpeg, through the external library, so you should be able to encode directly in VVC in FFmpeg.
Finally, the Intel folks have added VVC hardware decoding through QSV.
LC-EVC, MV-HEVC
On the MPEG side, two new codecs were added to the codebase: MV-HEVC and LC-EVC.
MV-HEVC is the 3D version of HEVC, that is used in recent iPhones and in the Apple Vision Pro.
While technically, it’s not a new codec, but an extension of a codec, adding this to the codebase required large refactoring of the HEVC decoder (which got us to fix some longstanding HEVC bugs) and it required new APIs to manage Multi-Views in FFmpeg.
Please note that x265 should be able to encode in MV-HEVC now!
Very different from MV-HEVC, the LC-EVC decoder was also added to this release too.
LC-EVC is a layer enhancement codec, sponsored by V-Nova and called MPEG-5 Part 2. It adds layers to a main codec, to allow to improve it: you can use LC-EVC with H.264, HEVC or AV1 base layers.
Adding this codec was quite complex for FFmpeg architecture, because we don’t have many enhancement codecs, and this made us leverage the Stream Groups concept that we’ve added also for IAMF. I expect that we see more of those type of codecs, in the future.
The LC-EVC decoder is now in FFmpeg, and it depends on an open-source external library, that is BSD-licensed!
Thanks James and Anton for those 2 decoders!
xHE-AAC, IAMF
Lynne spent a large part of her past year to work on a native xHE-AAC decoder.
Again, this required large preparatory work to prepare this new decoder. A lot of refactoring of the AAC decoder was necessary to support USAC and other features that got us this new xHE-AAC decoder.
As we’ve not seen many xHE-AAC streams in the wild, we’re not 100% sure of the compliance, but we’re sure it will improve when more samples are available.
On the other side of the spectrum, the AOM IAMF decoder/demuxer, introduced in 7.0 got a lot of fixes and testing, so I’m now quite sure that the IAMF decoder is now production ready!
To finish on the audio side of things, support for LC3, the Low Complexity codec from Google, was also wired through the official external library.
Vulkan hardware encoding and other hardware news
On of the other major feature of 7.1 done by Lynne is the support of encoding through Vulkan.
I know, it feels like yet another hardware encoding library, but maybe this one will unify all the other ones, since it’s part of Vulkan standardisation.
FFmpeg now supports H.264 and HEVC encoding through Vulkan. AV1 will come soon, I hope. (Vulkan decoding was done in 6.1)
We also now have a D3D12 encoding for HEVC, after we added D3D12 decoder (Decoding of H.264, HEVC, AV1 and others were added in 7.0).
Misc
I would also like to mention, as the maintainer of libdvdread, that DVD seeking was added to FFmpeg 7.1, which makes the DVD playback complete.
And finally, the YUVJ pixel format is officially deprecated!
A method for YUV colorspace negotiation for codecs and filters was added in this release to fix this problem.
The topic of YUVJ removal has been a recurring one for the last 5 years. I’m very happy that this is solved!
And like with dav1d, a large number of optimizations for ARM64 and RISC-V were added to accelerate FFmpeg decoding on non-x86 platforms. Your mobiles will love it :)
Final words and LTS
This is a very strong release, with a lot of important features and continuing the work that was started for 7.0, which was a huge release because of the multi-threading development.
I don’t recall an FFmpeg release with this many codecs added in a single release.
This release is more than 2700 commits, and the diff is around 211kLoC!
Thanks so much to Andreas, Michael, Anton, James, Rémi, Lynne, Niklas, Zhao, Nuo Mi, Ramiro, Marvin, Haihao and Martin, who are our top contributors for this release!
As for me, I consider FFmpeg 7.1 an LTS release, like the 5.1 release, so it’s a good release to bet on!
I hope you Enjoy this release!