You can spot their glowing red entrances all across The Golden Isle.
To enter a Vault, jump down into the hole.
Once you have completed the puzzle in the Vault, it will change colour, depending on whether it is fully complete or not.
Red - Incomplete.
Once a Vault entrance turns white, it will be sealed and you cannot enter it again.
You can leave a Vault at any point by returning to the glowing gold column where you started and selecting the Exit Vault of Tartaros button.
If you have any other questions, please get in touch.