Johannesburg's train station, Park Station, is in Braamfontein, at Leyds and Loveday streets. The famous, luxurious Blue Train, which makes regular runs to Cape Town, departs from here, as do Shosholoza Meyl trains to cities around the country, including the Trans-Karoo to Cape Town, the Komati to Nelspruit in Mpumalanga, and the Trans-Natal to Durban. Many of these trains have overnight service. They vary a lot in terms of comfort levels and price, and the more expensive ones such as the Blue Train are often booked up far in advance (though it's always worth a try if you want to make a reservation on short notice).
Locally, you can catch the Gautrain from O. R. Tambo, Pretoria, Sandton, or Rosebank to Park Station (see).