Desmond knows Lucy's leaving, and he wants to follow her, but he also knows it's not something he can do. This is going to just keep going in a circle for them, every time things start to get good. He glances at the others, guilty ashamed side looks, and walks the opposite way. One of them, Ashley no doubt, will go see to Lucy. Desmond just leaves the shrine by himself.