If the server is located closer, you can expect faster data transfer speed. That means, your website will load quicker if the server is located closer to you. Generally, visitors don’t like to wait long until a website gets loaded. If a website takes more than 3 seconds to load, your visitors will likely leave your site without looking at what you have to offer.
Server location can definitely be important depending on what you need it for. If you want to see faster response times due to lower network latency then choosing the closest server is needed. Please note most providers have multiple datacenter locations so most likely a server is available to you pretty close.
As mentioned by Efe Agbontaen you can also set up your website to use a Content Delivery Network.
This is yes and no question. If for example you have a company website and you only work in your country then is pointless to have load balancing because your audience most likely will be from your country, however, If your business is international and you receive high traffic then is highly recommended to use load balancing server to spread the visits and increase the traffic
It depends on the product you are going to host on the server. Let me put it this way, the farther your server is from the user, higher the ping will be for that user. But in most cases, it doesn't matter where your server is located unless you have high traffic from one specific region. If you have high traffic from all over the world, someone already suggested you to use CDN which will be the right choice in this case. If you want to inquire about something specific, do not hesitate to ask.