Financially, well it depends on your personal circumstances and when and where you are diving. If you will only be partaking of one or two dives on a family holiday and the hire charge is minimal, then that would be far cheaper in the long term than buying. But if you are diving at home most weeks, hiring could be at best impractical, at worst costing a small fortune.
There are many benefits other than financial savings in owning your own diving gear. Not having to share regulators with unknown previous diving guests is just one of them! More importantly, you can become familiar with your own diving gear, which gives a confidence that you might not build up if you are constantly hiring different gear whenever you dive.
Personally, I would recommend buy in just smaller pieces of gear every now and then. Remember that the bigger items such as BCDs and regulators not only need to be purchased, but will also incur maintenance costs as well as potentially being heavy to carry in luggage. Even though I own my own BCD and regulator set, I do not always take it away with me if I am only expecting to partake of the odd dive or two.
So where to start? If you are setting out buying your diving gear then a good point to start with is your mask and snorkel. These are very personal items and do not cost too much. The mask must be a perfect fit, so owning your own could make some dives far better than if you are using ill fitting hire gear. I would go as far as saying that you should probably buy these items even before you commence your basic training so that you become used to them.
The other benefit of your own mask and snorkel is that they are not limited to scuba diving. Once you are happily settled into your holiday resort you can kit up in these and have a go at snorkelling. This can be the opportunity to practise mask clearing exercises, even if only within the hotel's swimming pool. If there are a few days before you commence your four open water diving sessions of the course, say while you complete the theory, then it is at least a step in the right direction.
After that you might like to look at the safety equipment and accessories such as diving torches and knives. Your own torch can be a great benefit on a night dive and a knife, once you have finished your training, can be for prodding and signalling on dives.
Once you are settled into your diving then you might like to look at the more expensive and larger items of gear. Familiarity with your gear might just be a great gain one day.