 What if the ones who own land (maybe even the airspace) surrounding you don't let you leave or enter via roads?

That would be confinement. You can't just build a wall around someone and say, "Hey, I haven't harmed you or your property." You are still violating their property rights, more specifically, their right to access property.