The caves are upriver from LP about 25km by road, slightly less by river. You can head up in a tour on the river from LP // prices about USD10pp shared between 5 people - takes quite a few hours. You can hire a motorbike and get there on your own - daily hire is reasonable plus it gives you the freedom to explore. Get your hotel or accom to draw a map to follow as English is not common in rural Lao, and 25 km out of LP is definitely rural!
From land the caves are across the river so you will have to board a boat to visit from there. Under USD3 for that, and a few restaurants to get into shade and a drink make that approach well worth while. almost 10 years ago