My desktop has an 80GB primary for the OS, and a 2TB secondary for data.
Similar situation at work, where most workstations only have a few hundred GB for the primary boot disk, and have a 2TB or 3TB secondary data drive.
Programs are still typically installed on the primary, which generally makes sense, because many of them are integrated with the OS in terms of registry entries and shell extensions and such. If you replaced the OS, you'd probably want to re-install the stuff in "Program Files".
Most of the user data at work is either large static data sets, or projects that other people might need to work on or have access to, so stuffing them in a private "MyDocuments" folder doesn't make much sense. Hence no need to move that either.
In my workplace setting, there is no need to really replicate the structure of any of the typical system folders on the secondary drive. If you wanted to though, I believe their location is controlled by expandable environment variables, so they should be customizable. I've never tried it though.
In short, yes, use the smaller for your OS, but I probably wouldn't bother moving things like "Program Files" to the second harddrive. Just dump your own files there however you feel like it. Or possible copy programs there that don't need installation. (Unzip and run, like the OP2 download).
