Adding resources is often the cheapest and fastest way to improve performance. Especially with something like response time.
But is it the best?
That depends. Tracking down and optimizing performance problems in both server configuration and code can be expensive, highly specialized, and time consuming tasks.
And how will you know?
You must start measuring. Measure response times, measure time to first byte, measure amount of data transferred. Automate these tests and measurements using some of the many tools out there for server monitoring. Graph the results.
Finding the right balance between more resources (cores, memory, ...) and server/code optimizations (programmer/admin hours) is up to you. I suggest you try a bit of both.
And remember, if you aren't measuring than there is no point trying to optimize.