If this feature is getting in the way, you can adjust it.
First, go to the Apple menu and select System Preferences.
Next, click on the "Security & Privacy" icon.
Lastly, make the adjustments you want. This could include lengthening the delay so it is 15 minutes or even an hour instead of 5. You could even turn it off entirely.