The back and forth between one voiced character and one not just completely ruins the immersion

Feel like I'm back playing mmos again where only one guy has a mic in the dungeon. I know switch to another language or maybe mute the dialogue altogether but it feels like they left the best version(imo) feeling incomplete...

It's frustrating because otherwise I am LOVING the new planet and everything about it I just wish I could experience it properly.