Here are my top 3 of what to do in Liepaja:
Visit Valhalla Wine & Coffee! This really cozy bohemian cafe/bar I found by coincident on Google maps (as the city itself isn't very known among tourists it's hard to find any good blogs or recommendations of where to go etc). Here you can sit and chill in their lovely basement lounge, play with the cat and dog who lives in the cafe or just enjoy one of their delicious drinks, beers or cakes (or food!). And meanwhile waiting for your food you can chill in one of the many different couches. Ps: try their local ginger cider! In Latvia they also have many nice local cider breweries.
Visit the bar called Miezis & Kompanija that's in the same area as Valhalla. Here you can try hundreds of different local craft beers. I just wish my tolerance would be better and I could have tried many more haha. In this area all the bars are nice, kind of hipster-cozy. If we would have had some more time I would definitely have been going to all of them. Another must-to-visit is the Greek street-food place "Greeks" next doors! Their halloumi is just AMAZING!
Take a local bus a few kilometers up north to the old Soviet town Karosta. Historically this place is very interesting as you can see the influence by the Soviet Union in the area. There are old ruins of the Soviet sentries and fortress protecting the coast from enemies. We walked a few kilometers out on the old pier called "Ziemelu mols".