The older gear had no issues with 4.0 quad; it's the chipset in newer AVRs that no longer know how to process 4.0 files via HDMI. The file has to be at least 5.0 to play in surround. Your 4.0 files from all of these Rhino Quadio discs will only be heard in 2.0 stereo because that is how the 4.0 is being interpreted.
Which Denon models are you referring to? Anything latter day (after 2011) is almost certainly not going to have analog (6CH IN) inputs and the player you have (which model?) likely lacks the analog outputs to feed an AVR's external input.
As suggested, the material can be augmented with a silent channel (or two) to fool the modern chipset. That would be discussed elsewhere.
That or a potential change in playback hardware.