Turning back is never the wrong decision. As you say, its too beautiful a part of the world to be not enjoying yourself!
For what its worth, it doesn't look to me like you reached the trickiest bit. The summit itself is off to one side of the main ridge and involves a definite few moves of upward scrambling/climbing which feels a little insecure.
See this post from Sgurr with an excellent image demonstrating it.
https://www.walkhighlands.co.uk/Forum/viewtopic.php?p=334479#p334479The cairn in your picture is, I think, on another sub-summit.