when you lease a dedicated server, you are leasing and entire physical server. The entire server (hard drives, ram, cpu, raid card, motherboard, everything is used by only you. This gives you better isolation and no risk of another account on the server, affecting your account.
A VPS is a "Virtual Private Server". It is a virtual server, running on a physical server. VPS's share the physical resources with the other VPS's on the server. A VPS is usually better then regular shared hosting, but not as good as a dedicated server. However, this also depends on the resources included in the plan you choose.
i.e. you can get a small VPS, that has less resources then you would have on a quality shared hosting account. You can also find older/slower dedicated servers that cannot perform as well as a VPS on newer hardware.
The other thing to consider is server management. If you know how to properly setup/secure/manage a server, then your all set. If not, you will need to get a managed VPS/server and have your host handle these tasks for you.