Reaper for immersive up-remixing

[zeerround]

Continuing this thread: https://www.quadraphonicquad.com/fo...ormat-by-samsung-and-google.36964/post-827888 here under upmixing, I've finally made time to climb the learning curve on using Reaper for immersive up-remixing.

Previously I've been using plogue bidule, and trying to make it look and act more like a traditional DAW for remixing, and had only used Reaper, in batch mode, for VST based upmixing. But now I understand "routing" in Reaper, so along with the Reaper native ReaSurroundPan plugin:

have made a 7.1.4 up-remix "session template" in Reaper:

This has tracks for all the possible stems you can make with Lalal.ai:

Drums
Bass
Elec Guitar
Acoustic Guitar
Synth
Piano
Strings
Wind
Lead Vocal
Backing vocal

and via stem subtraction

Vocal ambiance
"Other" (anything in the original stereo that didn't make it to one of the separated stems)

the upmix part is I upmixed the "other" stem and the vocal ambiance stem to 5.1, for placement in the 7.1.4 bed, and heights (with a virtual center high) respectively.

Each track has an instance of the ReaSurroundPan plugin for panning the stereo or 5.1 input stem to 7.1.4 output space.

Also, this template uses the EAR Production Suite plugins, such that you can monitor the 7.1.4 output with binaural headtracked headpones, and check the stereo mixdown. The template is also setup to make outputs for eclipsa/iamf audio encodings (but only Samsung 2025 TVs and Soundbars can decode at this time).

I also used Reaper/made templates for the stem subtraction, and a simple template for playback, over binaural headtracked headphones, of a single 7.1.4 wav file.

Note that all of this could have been for 9.1.6, or any other immersive or non immersive surround format.

Reaper is a very powerful (now that I understand the routing!) and low cost ($60 us) DAW and many upmixers here may already have it for batch jobs with the Penteo plugin, etc.

For testing I chose "New York Minute" Eagles Live MTV 1994, as a track that had all the instruments so I could use all the possible stem separations in Lalal.ai. Soprano Sax being a "Wind" instrument.

I'll post more about the binaural headtracking, and one way to monitor 7.1.4 live via HDMI to an AVR, as the point of all of this (besides my personal use) is to build up a low cost ecosystem for up-remixing to immersive surround formats. At the moment Dolby Atmos, DTS:X, Auroa 3D encoders are still out of "low cost" reach but we starting to have some open source encoders like Mpeg-H and eclipsa/IAMF (but need the decoders built into consumer gear).

Some of us do have Dolby Atmos and/or DTS:X encoders, so I guess as a community we could get things encoded in the meantime. Oh, and also there is Dolby Atmos for streaming quality (but not TrueHD quality) encoding possible for pennies a track via AWS cloud.

Progress!

PM me if you want the Reaper templates.

You can build this head tracker (<$20 US): https://github.com/trsonic/nvsonic-head-tracker and I can share the 3d print file I used to mount it on my headphones, as well.

 

[zeerround]

Also, on the surround monitoring via headtracking and binaural headphones, there are more expensive / higher quality solutions:

Some of them are:

Waves NX - $36 US (I suggest using a web cam for tracking vs. the optional headtracker) but this is limited to first order ambisonics as an input. It's easy to encode 1st order ambisonics with free VSTs but 1st order is not great quality.​

​

dearVR MONITOR - $180 US (Should be able to use the <$20 headtracker I mentioned in the last post.​

​

Genelec has something: $$ subscription https://www.genelec.com/aural-id but it seems like you can't buy it at the moment​

​

And, of course, the most excellent Smyth Realiser A16 $4,695 USD (plus they want you to use very high end headphone. I chose Sennheiser HD 800 S)​

​

However I'll be interested to see how well the EAR Production Suite method works for other, as it's free. I'll also post about other methods, as time goes along.

 

[zeerround]

Oh, I forgot to mention there is also a Reaper template for playback 7.1.4 (12 channel wav file) from foobar2000 or other 12 ch. capable player to reaper and then Binaural with headtracking.

You will need the $15 US loopbe audio virtual audio cable: ps://www.nerds.de/en/loopbeaudio.html

There are various other virtual audio cables, but this one is 100% solid and works without effort.

For the monitor 7.1.4 live via HDMI to an AVR, solution I mentioned above, loopbe audio is also required.

 

[humprof]

Oh, I forgot to mention there is also a Reaper template for playback 7.1.4 (12 channel wav file) from foobar2000 or other 12 ch. capable player to reaper and then Binaural with headtracking.

You will need the $15 US loopbe audio virtual audio cable: ps://www.nerds.de/en/loopbeaudio.html

There are various other virtual audio cables, but this one is 100% solid and works without effort.

For the monitor 7.1.4 live via HDMI to an AVR, solution I mentioned above, loopbe audio is also required.

Amazing stuff, @zeerround. I suspect the answer is "No," but would the above have any implications, however oblique, for something many of us have long desired from foobar, namely HDMI pass-through for Atmos?

 

[zeerround]

Amazing stuff, @zeerround. I suspect the answer is "No," but would the above have any implications, however oblique, for something many of us have long desired from foobar, namely HDMI pass-through for Atmos?

Not passthru of already encoded audio, but live streaming, via window's embedded Dolby Atomos encoder, from a 12 ch wav file to an AVR, via dolby atmos, over hdmi.

You can do what you ask via VLC, however (not from foobar2000, as far as I know).

 

[zeerround]

Still on the Binaural Headphone thing, Auro 3D ofers a free VST3 that does Binaural monitoring, but lacks head tracking. The documentation on their channel order is wrong, and their head size units are goofy (for me inter ear distance = 14.32 cm or 0.65 on their scale):

You do have to have an ilok, however, but these days I think you can use a software ilok? If so, don't know what, if anything that costs.

Besides the other weirdness mentioned above, without head tracking I would pass on this one, unless it works for your head/ears better than others.

I'm going to re-check Auro 3D's pricing on encoding. I keep hoping for pay as you go, vs. subscription... hmm still $20 US per month.