Most people will say that more Ram is more important. However there is no educated way to know the answer to this, without knowing your site, what it does, and how many resources it uses.
If your currently using 50% of your CPU, adding more will not help. Same goes for RAM. If you are currently on a VPS or a dedicated server, take a look at your resource usage to determine what might be holding you back.
If you are on a shared hosting plan, there is no real way to determine how much your site needs.
The best bet is to get a VPS with the resources you think you need. The great thing about VPS's is they can be scaled up or down instantly. If you order on and find you need more RAM, just ask your host about increasing your RAM. If you need more CPY cores, just ask your host to add another CPU core to your plan. It's that simple!