hm its been ages ago since i did have that one.. last time was when i freshly gotten my 5080.
So question if it hasnt been fixed yet, have you reccently updated your GPU's driver?
-if not, make sure to update if you arent on the latest alrdy and never use a GPU driver version that is lower than .5 BECAUSE they all are bugged or have big problems..
-if yes, revert to the old driver
-Also if you alrdy done those things maybe the workaround works grab your "nvEncodeAPI64.dll" in Windows\System32\DriverStore\FileRepository (which needs advanced admin/security set up to get in) and put it into your obs plugins folder (obs-plugins\64bit) and try again
But also what i 4got to ask, did you reccently update obs? if yes that might be the highest chance for the cause (if your like me you got 3 obs versions one that always works then the save and then the one you update, so just revert and try ^^) )