Yes, since you're just doing a simple setup, you should have two IDE cables coming from the motherboard. Find the first one (labeled IDE0 usually). This should already be plugged in to your first harddrive. Hopefully this has two connectors on that harddrive end, if not, you'll have to buy a new cable (assuming you're using IDE1 for your CD drives--- you want to keep CD drives and harddrives on different IDE channels).
There are jumpers on the back of your harddrives. Since I don't know how old of a setup you have, I'll give you a way that will always work. Set the jumper of your main drive to master and the jumper on the spare (secondary, no OS) drive to slave. Plug the IDE cables in and power them up
