Set it to a nearby place instead, if you're concerned about the potential dangers if someone accesses your phone.
Every time you're heading home from an unfamiliar location, you always have to type your home address in Google Maps. It's like a routine by now, but that doesn't mean it isn't still a hassle.