I would think you’ll need to separate the loop from the display. Pseudo-code:
create a blank array
foreach // as you have it
append each file to an array as elelement ['name']
add the filemtime() information to that array as ['ftime']
end foreach
sort the array
loop through the sorted array to display
My guess on the sort would be more like
uasort($filearray, function($a, $b) {
if ($a['ftime'] > $b['ftime']) return -1;
if ($a['ftime'] < $b['ftime']) return 1;
return 0;
}