It's happening more and more. Medical professionals are being hurt, attacked, even killed while providing care. We go wherever medicine takes us. Caring for others is a part of who we are.
Let's learn to care about ourselves and our safety more. You are not obligated to go into any situation that you feel is unsafe for you. You do not have to deal with the aggressive visitor that has threatened you more than once. You do not have to deal with sexual harassment from your patient. You are not at work to be subjected to physical abuse from your patients. Your safety concerns are valid. You have every right to demand a safe work environment.
If you need to utilize security then do so. If the nursing staff has to be changed because the patient only harasses a particular nurse, then make that change happen. Report threats and take threats that are reported seriously.
Furthermore, be aware of your surroundings. Know where you are in your room in relation to the door. If the situation calls for it, take a second nurse in with you as your "back up". In some hospitals, security can be used to sit with the patient, if the patient is deemed a danger to the staff (the hospital I am at currently does this).
You are important. Know that your safety is important too.