You can easily spend several hours hitting the stores, boutiques, and historic sites of the Faneuil Hall and Quincy Market complex. On Friday and Saturday join crowds at the Haymarket farmers' market (wear good walking shoes, as the cobblestones get slippery with trampled produce).
Allow two hours for a walk through the North End, longer if you plan on dawdling in a café. This part of town is made for strolling, day or night. Many people like to spend part of a day at Quincy Market, then head over to the North End for dinner—the district has an impressive selection of traditional and contemporary Italian restaurants.
Be cautious in the Government Center area late at night. The North End is a relatively safe neighborhood; you'll have plenty of company day or night.