To get your podcast in all the podcast players you must store the mp3 files of the podcast somewhere. The RSS feed then tells the podcast player where to find those mp3 players. To store an mp3 file on the internet, you need a podcast hosting provider.
There are over 50 podcast hosting providers out there today. Picking one can be daunting. Here's a simple overview and breakdown and comparison of 30 of these providers.
There are a few key things to consider when picking a provider:
You obviously want your listeners to get the show quickly when they go to play it. So speed and reliability can affect your listener experience in that way. You want people to download the episodes quickly and the server should be up for them to download it.
Cost is going to be between free - $40 per month. The free hosting providers don't always have good reliability. They frequently end up closing down and their terms of service can be scary. It's possible to store your mp3 files on Github and Archive.org to get free hosting but this is a little more advanced. The more technical you are the more creative you can be. Paying $5-$15/month is fair.
You want to know who's listening to your show, so most providers will give you access to analytics to see the number of downloads.
When you run into problems with your podcast you may need help. So having a podcast hosting provider that has good support can be a lifesaver.
Hosts can also have additional features such as Wordpress plugins, ability to publish from your smart phone, give you a free website, and more.
Currently, the top hosting providers in no particular order are:
Any of these are great hosting providers.