Hong Kong's streets may seem utterly chaotic, but the public transport system is not. Be sure to purchase a rechargeable Octopus card, which can be used on all buses, trains, and trams, the ferry, and even to make purchase at vending machines, convenience stores, fast-food restaurants, and the racetrack.
The quickest and perhaps safest way to travel is with the ever-reliable MTR (underground railway), which links to most of the areas you'll want to visit. There's no timetable because trains run so frequently. Signs and announcements are in both Chinese and English, and posted maps help visitors navigate outside the stations, too.
Although you can cross the harbor on the MTR, the Star Ferry is a cheaper ride, with the added bonus of being able to enjoy the fantastic harbor views during the 10-minute journey between Hong Kong Island and Tsim Sha Tsui. To get to the outlying islands, take a sampan or hire an air-conditioned junk for the day.
If you prefer street-level travel, the city's air-conditioned double-decker buses can take you anywhere, provided you know which number and route to take. On the northern side of Hong Kong Island, you can also take the tram (listen for the distinctive "ding-ding"), a fun and inexpensive way to get from one side of the island to the other—and it's the same route that the MTR follows, so you should be able to walk to an MTR station from any tram stop between Sheung Wan and Shau Kei Wan.
If you do get lost, you can always hail a cab. Prices are reasonable if you're not traveling too far, and tipping is not required. Not all drivers are willing to cross the harbor, though, so be sure to ask before getting in, and, unless it's a designated cross-harbor vehicle, expect to be charged for the return-trip toll as well.
Perhaps best of all, Hong Kong is a city that's easy to explore by foot. In Central, use the covered walkways that run from buildings in the business district all the way to Mid-Levels, thus avoiding stoplights, exhaust fumes, and weather conditions (but not crowds). The same can apply to the pedestrian overpasses all around the city.