If he can remove it, it's not really doing it's job anymore, is it?
We still shut the gate at night, because K has to pass the stairs to get from her room to ours or to the bathroom - her door comes out right at the top of the stairs, so if she isn't quite awake and gets a little off-course, she could fall. Otherwise, I only close it if Rachel is playing upstairs. I think we stopped using it for Katie when I had seen her walk down enough times that I was comfortable with her stability.
Don't worry about what your friend says, because it's not her problem if there's an injury, kwim? I have a friend who inherited K's bedrail when we took it off, for a child 2 yrs older than K! But she still was falling out of bed sometimes, just an active sleeper I guess, so she needed it even though my younger child did not.