@braddo the easiest way to upgrade K2 is to use an in-place upgrade. But there is a limitation where you cannot upgrade the current OS and DB. If you want to upgrade the OS along with its DB, then you must do it on a new server (a staging server). More or less the step-by-steps that you informed are true and can be concluded as follows:

1. Prepare the staging server and install it with the latest OS and DB (if required)
2. Give the server name uniquely
3. Backup the previous K2 Five 5.6 DB
4. Restore the DB to the staging server
5. Copy K2 Five installation to the staging server and run K2 Five 5.6 setup manager from a staging server
6. Refresh the K2 server name and the DB connection 
7. Execute the K2 setup manager installation
8. Verifying K2 configuration analysis and making sure K2 workspace, management and designer can be accessed with the new staging server name
9. Deploy a custom code or library (if exists)
10. Testing the applications (make sure all are running well)
11. Update K2 DB in  K2].nServer].SWorklistHeader] table and update the url value from “Data” column and change it from FQDN K2 server name 5.6 into a new staging server. Example: change it from https://uK2_5.6_OLD_FQDN]:443/Runtime/Runtime/Form……. into https://lK2_5.6_NEW_FQDN:443/Runtime/Runtime/Form…...
12. Testing the applications
13. Execute K2 Five 5.7 upgrade
14. Repeat step 8 (after upgrade)
15. Repeat step 10 (after upgrade)
16. Finally, shut down the previous K2 Five server 5.6
17. Shutdown all K2 services (the staging server)
18. Disconnect the staging server from the domain
19. Rename the staging server to the previous K2 Five server 5.6
20. Join to a domain
21. Verifying the DNS (deleting the staging server hostname) and creating a CNAME if required
22. Run K2 Setup Manager again and refresh the hostname
23. Verifying K2 configuration and analysis (make sure all the K2 services has been turned on/started)
24. Testing K2 workspace, management and designer
25. Repeat step 11 and change it back to the old (K2 server name 5.6)
26. Lastly, test the application again

Note: When you do the upgrade and test it on a new server and want to change the server name then it would require the effort to disconnect and connect the server from the current domain, run the K2 Setup Manager, refresh the hostname and finally update the K2 DB as lies on step 11 (this is to update the previous url to a new url). 

We have tried this setup before, but it failed every time. The only difference between what we did and what you did is that we didn't remove the host from the domain before rejoining it. I wonder if that's why it failed.