Ok, I figured it out. I feel like an idiot because it's so simple. I just created two similar flows that do what you said but one was a when trigger and I reversed the variables for the two flows.
Quick question for home mode does it trigger for you every time a person enters? In other words if I'm home and my wife arives it still triggers home mode. I don't think it's a big deal, but figured if I could solve that it'll be perfect.