The world famous Ha Long Bay is worth the hype.
Halong Fancy Hostel: another standard Vietnam Hostel. Good food, cheap beer, and friendly staff who know the area so can help you organise trips to the bay.
The Bay: I got a day boat tour through the bay, stopping at various islands to go through caves, swimming, and then kayaking in a lagoon within one of the mountains. Also gave us loads of (sea)food that was of course delicious.
Quang Ninh Museum: a local history museum, it's very big so may take some time, but is certainly interesting with many examples of real equipment/clothes used in the past.
Poem Mountain (Bai Tho): the best view of the bay. To get onto the mountain you have to pay to go through somebody's house, which backs onto an old trail, where you then climb up a small cliff to reach a path. After that it's just stairs through some overgrown jungle, but it's worth it. It's not an official business and I'd recommend a bit of fitness, but it's a brilliant view. If you go for the sunset like I did, take a light. The stairs are hard to navigate in the dark.