And of course as soon as I’d posted this I realised it was an issue with the CloudFront configuration. Make sure you have the Behavior set to /sub-folder* and not just /sub-folder.
It seems the documentation is configured correctly. I tried accessing and navigating without issues. Feel free to reach out if you have any other questions.