You should always keep up to date with new releases such as Apache, PHP MySQL etc...
Right now my servers run Apache 2.2.24 With PHP 5.4.13 as it just got out of experimental phase. I tend not to run anything experimental.
Updates are released for a reason, however some scripts found on the internet tend to use depreciated functions as the developer(s) do not update their code, which will result in errors obviously
This is another reason why most webmasters like using platforms like Wordpress, Drupal etc... as they constantly keep up to date with the times