This is certainly not the behavior we expect.  It sounds to me like some innapropriate javascript or Skuid Actions are being run whenever rows change on the model,  or rows are marked for deletion.  Look at Model Actions on your Attachment Model, or look at any custom javascript snippets to see if any of these would be responsible for this behavior. 

Yes, I had a action to re-query the model when model was cancelled. It appears this is what was causing the issue. Thanks

Glad we found it!