There's not a limitation on the number of controls on a form on On-Prem Nintex, but there's a limit on how many controls can be in other controls (because of XML depth). You will also face issues if the form is too large. I've had form that were 10mb, but I wouldn't dare push further.

Couldn't find any hard limits for On Prem other than the limits described by SharePoint itself. This post has some of that information: https://community.nintex.com/t5/Nintex-for-SharePoint-Forum/How-to-Improve-Form-Performance-and-Improve-User-Experience/td-p/84847 

However, for Nintex Cloud, it is capped at 2000 Controls: https://help.nintex.com/en-US/nwc/Content/Overview/Limits.htm