Hey Taako, my friend.
Have you tried changing the file name after you change the fonts?
I believe that I experienced the same type of problem when I first started using aegisub years ago.
Every time I make a change to fonts, I save it with a new name (I add dates to help me keep track of edits.)...