Jamie told me to use chroot to log in into the chroot jail of debian os that are exported for the FEs and install ntp there. I took following steps at the end of which, all FEs have NTP synchronized now.
Edit Tue Aug 24 10:19:11 2021:
I also disabled timesyncd on all FEs using sudo systemctl disable systemd-timesyncd.service
I've added this wiki page for summarizing the NTP synchronization knowledge.