To be fair, this is not really about what pronouns to use when talking to someone, but, rather, what to use when talking about someone, in third person.
Still, how that works will depend on language and in many languages grammatical gender is not determined by pronouns alone.