Reaper for immersive up-remixing

[zeerround]

Continuing this thread: https://www.quadraphonicquad.com/fo...ormat-by-samsung-and-google.36964/post-827888 here under upmixing, I've finally made time to climb the learning curve on using Reaper for immersive up-remixing.

Previously I've been using plogue bidule, and trying to make it look and act more like a traditional DAW for remixing, and had only used Reaper, in batch mode, for VST based upmixing. But now I understand "routing" in Reaper, so along with the Reaper native ReaSurroundPan plugin:

have made a 7.1.4 up-remix "session template" in Reaper:

This has tracks for all the possible stems you can make with Lalal.ai:

Drums
Bass
Elec Guitar
Acoustic Guitar
Synth
Piano
Strings
Wind
Lead Vocal
Backing vocal

and via stem subtraction

Vocal ambiance
"Other" (anything in the original stereo that didn't make it to one of the separated stems)

the upmix part is I upmixed the "other" stem and the vocal ambiance stem to 5.1, for placement in the 7.1.4 bed, and heights (with a virtual center high) respectively.

Each track has an instance of the ReaSurroundPan plugin for panning the stereo or 5.1 input stem to 7.1.4 output space.

Also, this template uses the EAR Production Suite plugins, such that you can monitor the 7.1.4 output with binaural headtracked headpones, and check the stereo mixdown. The template is also setup to make outputs for eclipsa/iamf audio encodings (but only Samsung 2025 TVs and Soundbars can decode at this time).

I also used Reaper/made templates for the stem subtraction, and a simple template for playback, over binaural headtracked headphones, of a single 7.1.4 wav file.

Note that all of this could have been for 9.1.6, or any other immersive or non immersive surround format.

Reaper is a very powerful (now that I understand the routing!) and low cost ($60 us) DAW and many upmixers here may already have it for batch jobs with the Penteo plugin, etc.

For testing I chose "New York Minute" Eagles Live MTV 1994, as a track that had all the instruments so I could use all the possible stem separations in Lalal.ai. Soprano Sax being a "Wind" instrument.

I'll post more about the binaural headtracking, and one way to monitor 7.1.4 live via HDMI to an AVR, as the point of all of this (besides my personal use) is to build up a low cost ecosystem for up-remixing to immersive surround formats. At the moment Dolby Atmos, DTS:X, Auroa 3D encoders are still out of "low cost" reach but we starting to have some open source encoders like Mpeg-H and eclipsa/IAMF (but need the decoders built into consumer gear).

Some of us do have Dolby Atmos and/or DTS:X encoders, so I guess as a community we could get things encoded in the meantime. Oh, and also there is Dolby Atmos for streaming quality (but not TrueHD quality) encoding possible for pennies a track via AWS cloud.

Progress!

PM me if you want the Reaper templates.

You can build this head tracker (<$20 US): https://github.com/trsonic/nvsonic-head-tracker and I can share the 3d print file I used to mount it on my headphones, as well.

 

[zeerround]

Also, on the surround monitoring via headtracking and binaural headphones, there are more expensive / higher quality solutions:

Some of them are:

Waves NX - $36 US (I suggest using a web cam for tracking vs. the optional headtracker) but this is limited to first order ambisonics as an input. It's easy to encode 1st order ambisonics with free VSTs but 1st order is not great quality.​

​

dearVR MONITOR - $180 US (Should be able to use the <$20 headtracker I mentioned in the last post.​

​

Genelec has something: $$ subscription https://www.genelec.com/aural-id but it seems like you can't buy it at the moment​

​

And, of course, the most excellent Smyth Realiser A16 $4,695 USD (plus they want you to use very high end headphone. I chose Sennheiser HD 800 S)​

​

However I'll be interested to see how well the EAR Production Suite method works for other, as it's free. I'll also post about other methods, as time goes along.

 

[zeerround]

Oh, I forgot to mention there is also a Reaper template for playback 7.1.4 (12 channel wav file) from foobar2000 or other 12 ch. capable player to reaper and then Binaural with headtracking.

You will need the $15 US loopbe audio virtual audio cable: ps://www.nerds.de/en/loopbeaudio.html

There are various other virtual audio cables, but this one is 100% solid and works without effort.

For the monitor 7.1.4 live via HDMI to an AVR, solution I mentioned above, loopbe audio is also required.

 

[humprof]

Oh, I forgot to mention there is also a Reaper template for playback 7.1.4 (12 channel wav file) from foobar2000 or other 12 ch. capable player to reaper and then Binaural with headtracking.

You will need the $15 US loopbe audio virtual audio cable: ps://www.nerds.de/en/loopbeaudio.html

There are various other virtual audio cables, but this one is 100% solid and works without effort.

For the monitor 7.1.4 live via HDMI to an AVR, solution I mentioned above, loopbe audio is also required.

Amazing stuff, @zeerround. I suspect the answer is "No," but would the above have any implications, however oblique, for something many of us have long desired from foobar, namely HDMI pass-through for Atmos?

 

[zeerround]

Amazing stuff, @zeerround. I suspect the answer is "No," but would the above have any implications, however oblique, for something many of us have long desired from foobar, namely HDMI pass-through for Atmos?

Not passthru of already encoded audio, but live streaming, via window's embedded Dolby Atomos encoder, from a 12 ch wav file to an AVR, via dolby atmos, over hdmi.

You can do what you ask via VLC, however (not from foobar2000, as far as I know).

 

[zeerround]

Still on the Binaural Headphone thing, Auro 3D ofers a free VST3 that does Binaural monitoring, but lacks head tracking. The documentation on their channel order is wrong, and their head size units are goofy (for me inter ear distance = 14.32 cm or 0.65 on their scale):

You do have to have an ilok, however, but these days I think you can use a software ilok? If so, don't know what, if anything that costs.

Besides the other weirdness mentioned above, without head tracking I would pass on this one, unless it works for your head/ears better than others.

I'm going to re-check Auro 3D's pricing on encoding. I keep hoping for pay as you go, vs. subscription... hmm still $20 US per month.

 

[austinsignal]

Sounds like you're having fun! I use Reaper every day to mix ATMOS. Incredible DAW, the routing capabilities are without peer, and the visual/operating customization abilities are endless. It's very popular with the sound-design engineers due to its incredible power, customization abilities, and the ability to run 100's of tracks efficiently.

As you can see from the little video, I'm using a custom skin, so it looks a little different than your version, but underneath it's the same thing. In case you're unaware, there are libraries of scripts and extensions available, the most popular of which are the SWS extensions. If you're going to do meaningful stuff with Reaper, make sure to download that package. One little example, when I drag audio files into tracks into my ATMOS template, Reaper examines the name of each track and color-codes them appropriately - all drums and percussion colored the same way, all guitars a different color, etc.

From looking at your screenshots, you're mixing into the bed - which is totally fine. But you can also use dynamic object pairs for the tracks with the Dolby Panner. One of the big advantages to using dynamic objects vs the bed is that for headphone listening, you have the ability to adjust the spatial distance for each track. This is done in the Dolby Binaural settings plugin. Each track can be set to near, mid, or far. When mixing, you can adjust these parameters for each track to get some nice separation and width in headphones. If you're mixing exclusively into the bed, you don't have this ability. Mind you, the adjustments only work with Dolby Binaural renders, not with Apple Spatial.

If you're looking for live headphone monitoring with Spatial and headphone tracking, try Sphere.

Reaper ATMOS mixing

J

 

[fetchmybeer]

Oh wow. Great stuff. Admittedly, I have not gotten my head around all the routing stuff in Atmos. I have a general understanding, but with limited time to actually sit down and figure it out, it just hasn't happened. I ended up getting Fiedler Audio's Dolby Atmos plugin (when it was on sale), which simplifies a lot of that.

 

[zeerround]

This is all great info! I will dig into the scripting stuff. But just to clarify, I wasn't going for "Dolby Atmos", since most hobbyists can't afford it or get a license even if they could.

I was anticipating Eclipsa/IAMF as a free alternative. Or maybe there are enough up-remixers here that we could do atmos or DTS:X encoding for each other (or use QWS media encoder for streaming quality atmos)?

Maybe as a community we could justify the $20 per month for Aura 3D encoding "service"?

But yeah (especially for atmos 9.1.6) Reaper seems way more strait forward (and economical) than pro tools. While I have Pro tools for both windows and mac, I stopped paying for maintenance because I don't use it often enough (as a hobbyist) to justify the cost.

 

[zeerround]

Oh, re: "If you're looking for live headphone monitoring with Spatial and headphone tracking, try Sphere." OK I will check it out.

I'm trying to get a free demo from dearVR, but their web form seems broken (sent email to support).

I also ordered a pair of Beyer DT 770 Pro 250 ohm cans, because 1) I wanted some closed back cans as an alternative to the Sennheiser HD 800 S cans I use on my realizer A16, and 2) I wanted something a little higher quality, more bass response that the Sony MDR 7508s I've been using for my binaural headphones/head tracking testing.

I'll add Sphere to the list to test, but it's 2x the cost of dearVR, for a perpetual license, and I don't think hobbyists would want/could justify the subscription.