Yes - you are unreasonable!!
Even if it is to drive 1/4 mile on a public road, the other person needs to be comfortable, reach the controls easily, and SEE in the mirrors.
Their safety, and the safety of others!
Suck it up, alter them back to where it suits you, and drive!