The problem will be related to iterating over a collection that is being moved about, and with enough pushing it could be solved but instead, at times like this it’s wise to know when to give up.
I’m going to be lazy, and effective. Here is a good jQuery shuffle plugin that you should use instead.
After you’ve loaded the shuffle plugin, you can use this to shuffle the list.
$(function () {
$('#slider ul').shuffle();
});