In the event source Event Behavior make sure that Allow Deletion of Events is checked.
In the Event Source On-Click Popup Properties, make sure that Show Default Buttons in Popup is checked.

Further detail to Matt’s response about the Event Source  - Event OnClick Behavior property: 

The first line of his answer is what happens if you are using the default popup. 
The second line of his answer is what happens if you choose the custom popup option.