this is a normal behaviour. Pause action will move the workflow to the SharePoint Timer Job. SharePoint Timer job by default runs every 5 minutes. This interval can be changed, but it is not recommended to do so to avoid overload.
Basically what you say is "workflow, wait until timer job runs and the time I specified has passed."
So if you think of a ping every 5 minutes your workflow will add itself to the queue during this period. Thus the workflow will start in between of 1 to 5 minutes, depending on when he hopped on onto the SharePoint Timer Job bus.
Does this make sense?