Why is the Refraction Affects Alpha Channel setting not working like I expect?

Refraction Affects Alpha Channel was moved to the Legacy settings because it only works in very basic cases. As soon as you use advanced transmission features like depth, single scatter, or have anything complex behind your glass or refractive objects, the legacy alpha method can’t accurately represent the way light bends and passes through the material. This is a technical limitation of the old alpha workflow, and why Redshift now recommends a more accurate AOV-based approach.
 
The recommended workflow now, especially for compositing scenes with refractive objects, is to use AOVs (render passes) instead of relying on the alpha channel for transparency.
 
Typically, you would render:

• An object mask for the refractive object
• A UV or Normals AOV to help distort the background behind the object
• Additional AOVs, like Specular or GI, to rebuild the final look

For assistance with AOVs, please visit the official Redshift AOV help documentation page >HERE<

In your compositing software, you can then use these passes to isolate and realistically distort the background behind the refractive object, and add back the necessary lighting and detail using the other AOVs. This method provides much more accurate and flexible results than the old alpha workflow.