I would say that you should expect a hurry up offense to be used at any time if the person on offense finds a mis-match they want to take advantage of. That mis-match often isn't found until you have run the play so you have maybe 1 second between the play ending and the players rushing back to the line - that isn't much notice and makes almost no difference in my opinion.
Calling it when you are going to use it for a whole drive, yes, that's a good time to call it, but for the odd play here and there it's not really worth it because by the time you have said you're going no huddle the players are on their way back to the line.
Defensive personnel should be chosen around the personnel sent out on offense with audibles to change coverages/back out of blitzes and time outs used by the defense if they feel unprepared for the next play.