If you're using Windows 7 or Windows 8 avoid installing RO in your "Program Files" folder, because those operating systems usually don't allow programs in said folder to change the registry keys. If you've already installed it in there you can simply cut and paste it into your personal folder, desktop or wherever you prefer.
Another problem is that heRO's current client is not compatible with the latest Setup.exe (blue ghostring icon), so you need an older version (orange haired girl icon) and might need to run it in compatibility mode and/or as admin like Ptah said.
I recommend you to check my installation guide
here and download the Setup.exe from step 4.1, also make sure that you choose the right video card or you won't see all the available resolutions.