Andrew,

There’s actually more than one way to achieve what you want. One way is to add actions on the Return Objects, which will allow you to define object specific actions. To do this, under the Return Objects tab, click on an object. You’ll then see a red plus icon next to a red x icon. You can click the red plus icon to add actions for that specific object type. Once configured, you’ll see your actions displayed next to the search results as the label text (or just an icon if you specified an icon).

Here’s another way, that I actually thought of first (because I forgot you can do object specific actions ). The benefit of this method is that you can click almost anywhere on the result to activate it, instead of just on an icon. Every record Id in Salesforce follows the same pattern - it begins with the three digit code associated with its object. For example, every Account record begins with ‘001’. You can see a list of standard object codes here. Even custom objects follow this pattern, although you won’t know their code unless you look at a record (but once you know it you’re good;  I’m pretty certain they don’t change once decided). I think you can take advantage of this in your branch action to determine the object type of the result you selected. Under the search result Select Actions tab (aka where you define actions for all results, as opposed to above where you defined actions per object) I was able to add a branch with the formula 

STARTS_WITH({{{Id}}},'001')

and could tell whether I clicked on an account or not (I just assumed that if it wasn’t an account it was an opportunity, my only other return object. If you wanted to check against more than two objects you’d need more branch actions and it would start to get more complicated). You can then add an action to update a field on row for your lookup field (as long as you’re only dealing with one row). In the update field action you can use merge syntax (e.g. {{Id}} ) for the value, because the select action is in context of a specific record. Let me know if you have any questions!