Exhausting Memory - Tried Fixing Loops, Still Does Not Work

I’m currently experiencing a memory usage issue - but I cannot figure out where. I’ve tried replacing some of my foreach loops with for loops or by issuing another query to the DB, but I am still gettting the same error - “Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 72 bytes) in <my directory> on line 109”. Can anyone provide some insight as to what may be causing the issue? Thank you!

	$participating_swimmers = array();	
	$event_standings = array();
	$qualifying_times = array();
	
	$sql = "SELECT DISTINCT(event)
			FROM <my database>
			WHERE sex = 'M'";
	$query = mysql_query($sql);

	while ($row = mysql_fetch_assoc($query)) {	
		$current_event = $row['event'];
		$select_times_sql = "SELECT time, name, year, team, time_standard, date_swum
							FROM <my database>
							WHERE sex = 'M'
							AND event = '".mysql_real_escape_string($row['event'])."'
							ORDER BY time ASC";
		$select_times_query = mysql_query($select_times_sql);
		while ($select_times_row = mysql_fetch_assoc($select_times_query)) {
			//Create array with the current line's swimmer's info
			$swimmer_info["time"] = $select_times_row['time'];
			$swimmer_info["name"] = $select_times_row['name'];
			$swimmer_info["year"] = $select_times_row['year'];
			$swimmer_info["team"] = $select_times_row['team'];
			$swimmer_info["time_standard"] = $select_times_row['time_standard'];
			$swimmer_info["date_swum"] = $select_times_row['date_swum'];

			//Create "Top 8" list - if more than 8 A cuts, take them all
			if (($swimmer_info["time_standard"] == "A") || ($swimmer_info["time_standard"] == "B")) {				
				//Check if there are 8 or less entries in the current event, or if the swim is an A cut
				if ((count($event_standings[$current_event]) < 8) || ($swimmer_info["time_standard"] == "A")) {
					//Add swimmer to the list of invites
					$event_standings[$current_event][] = $swimmer_info;
					
					//Keep only the identifying information about the swimmer
					$condensed_swimmer_info["name"] = $swimmer_info["name"];
					$condensed_swimmer_info["year"] = $swimmer_info["year"];
					$condensed_swimmer_info["team"] = $swimmer_info["team"];

					//Check if swimmers name already appears in list
					if (!in_array($condensed_swimmer_info, $participating_swimmers)) {
						//It is a unique user - add them to the list
						$participating_swimmers[] = $condensed_swimmer_info;
					}
				} else {
					//Add the qualifying time that did not fit into the list to a list of qualifying times
					$qualifying_times[$current_event][] = $swimmer_info;	
				}
			}
		}
	}

	//Sort each array of times in descending order

	arsort($event_standings);
	arsort($qualifying_times);
	$num_of_swimmers = count($participating_swimmers);
	
	while ($num_of_swimmers < 80) {
		$sql = "SELECT DISTINCT(event)
				FROM <my database>
				WHERE sex = 'M'";
		$query = mysql_query($sql);
		while ($row = mysql_fetch_assoc($query)) {
			$loe = $row['event'];
			$num_of_qualifying_times = count($qualifying_times[$loe]);
			$event_standings[$loe][] = $qualifying_times[$loe][$num_of_qualifying_times-1];
			 
			 //Keep only the identifying information about the swimmer
			$condensed_swimmer_info["name"] = $qualifying_times[$loe][$num_of_qualifying_times]["name"];
			$condensed_swimmer_info["year"] = $qualifying_times[$loe][$num_of_qualifying_times]["year"];
			$condensed_swimmer_info["team"] = $qualifying_times[$loe][$num_of_qualifying_times]["team"];			
			//Check if swimmers name already appears in list
			if (!in_array($condensed_swimmer_info, $participating_swimmers)) {
				//It is a unique user - add them to the list
				$participating_swimmers[] = $condensed_swimmer_info;
			}
			
			//Remove time from array of qualifying times
			unset($qualifying_times[$loe][$num_of_qualifying_times-1]);
		}
		$num_of_swimmers = count($participating_swimmers);
	}
	
	arsort($event_standings);
	arsort($qualifying_times);
	$sql = "SELECT DISTINCT(event)
				FROM <my database>
				WHERE sex = 'M'";
	$query = mysql_query($sql);
	while ($row = mysql_fetch_assoc($query)) {
		$loe = $row['event'];
			echo "<h1>".$loe."</h1><br />";	
	
			foreach ($event_standings[$loe] as $es) {
				echo $es["time"]."  ".$es["name"]."  ".$es["team"]."<br />";
			}
	}

I see you have two queries inside loops. You should try to avoid doing this. If you can get the queries out of the loops, you might resolve this issue. It will require a bit of a rewrite in how you have your setup, but it’s a better practice (or so I have been told by so many).

Like Cute Tink says, it’s the extraneous looping (especially that loop of swimmers < 80). You’re getting the list of events and looping through that list up to 82 different times. Depending on how many events there are, that can get extremely expensive.

What exactly are you trying to do with this list? I think I know (my son’s a swimmer), but I need to know for sure before I can try to figure out the logic you need…

I sent you a PM Dave, thanks!

OK, I’m answering here because it might be useful to someone else.

Sorry it’s taking me so long to get back to you - I’m absolutely swamped at the moment.

Is this the actual table structure or did you remove some joined tables for the sake of your question?


SELECT time
     , name
     , year
     , team
     , time_standard
     , date_swum 
  FROM <my database> 
 WHERE sex = 'M' 
   AND event = '".mysql_real_escape_string($row['event'])."' 
 ORDER BY time ASC

If it’s the same structure, how do you tell two swimmers apart if they have the same name? I can think of at least three cases where I know of pairs of people with the same name were on the same team(one went down the the spelling of their full middle name). How do you tell them apart?

Here’s my thought process for you, and hopefully it’ll push you in the right direction.

All the array searching and sorting is expensive and unneccessary if your tables are built correctly. In this case, I would create a temp table

Assumption: Your table structure is something similar to this:

[INDENT]Table: Swimmers,
Fields: SwimmerID, SwimmerFirstName, SwimmerLastName, TeamID, etc.

Table: Events
Fields: EventID, EventTitle, Gender

Table: Meets
Fields: MeetID, MeetTitle, MeetLocation, MeetDate

Table MeetEvents
Fields: MeetID, EventID, SwimmerId, Time, TimeStandard
[/INDENT]PseudoCode:

  • First Create a temp table that contains the A qualifying times

CREATE TEMPORARY TABLE #ChampionshipQualifiers 
 SELECT EventID
     , SwimmerID
     , Time
     , MeetId 
  FROM MeetEvents 
 WHERE TimeStandard = 'A'

  • Second, find the events which do not have eight swimmers in them. Loop through them and add top X swimmers for each event which haven’t already qualified in that event

 SELECT EventID
          , COUNT(*) AS SwimmerCount
   FROM #ChampionshipQualifiers 
 GROUP BY EventID
HAVING COUNT(*) < 8

then for each row in the loop (@EventID & @SwimmerCount are values that come from loop)


 INSERT INTO #ChampionshipQualifiers
 SELECT EventID, SwimmerID, Time
   FROM MeetEvents
  WHERE EventID = @EventID
    AND TimeStandard = 'B'
    AND SwimmerID NOT IN (SELECT swimmerID 
                            FROM #ChampionshipQualifiers 
                           WHERE EventID = @EventID) 
  ORDER BY Time
  LIMIT (8 - @SwimmerCount)  

  • Now you should have the top eight qualifiers for each event. Now you can get a count of how many distinct swimmers you’ve got

SELECT COUNT(Distinct SwimmerID) FROM #ChampionshipQualifiers) 

[LIST]
[]Now loop through all events and add your filler swimmers in. If you divide the swimmers left by the number of events to get the number to add per event, you could use almost exactly the same query used to to fill the eight) until you meet your target count.
[
]Then use this temp table to join back with the Event and Swimmer tables to fill in the “human readable” data.
[*]Once done, don’t forget to drop the temp table!!!
[/LIST]Hope that clears it up and gives you a direction to shoot for. If something here doesn’t make sense, please let me know.

NOTE: The queries might not be exactly right with the syntax (I jump from mySQL to Oracle to MS SQL all the time so invariably one creeps into the others), but the basic logic is there…

Wow, thank you for the awesome reply Dave! I’ve worked through your code and implemented it. But, I am having a small problem. I am trying to reach a maximum of 160 swimmers total - but for some reason, I keep coming up with 162. I cannot find where these two extra are coming from! I’ve attached my tables with data and my php file so you can see exactly what I’m doing.

By the way, the temporary table did not work as such. From the research I did, it seems MySQL doesn’t allow you to do two select statements on a temporary table in the same query - so I simply create a real table, and then drop it when I’m done running the script.

Thank you for all your help! If you could point me in the right direction on where my two extra swimmers are coming from, that would be great!

By outputting the count() each pass through the while loop, I discovered that the following INSERT statement is inserting 8 times each time its run - which, ends up totaling 162 because the previous pass is 156. (156 + 8 = 162).

		$insert_sql = "INSERT INTO d2_2011_cap_times_championship_qualifiers
					 SELECT SwimID, EventID, SwimmerID, Time, TimeStandard, DateSwum
					   FROM d2_2011_cap_times_meet_events
					  WHERE EventID = ".$row['EventID']."
						AND TimeStandard = 'B'
						AND SwimmerID NOT IN (SELECT SwimmerID
												FROM d2_2011_cap_times_championship_qualifiers
											   WHERE EventID = ".$row['EventID'].")
					  ORDER BY Time
					  LIMIT 1";

So, I know now where my problem lies. However, I’m not sure how to fix it - that insert statement is so handy and tidy! Can you point me in the right direction?

I’ve fixed it so it now only displays 160 swimmers, and now I am in the process of making sure the data is correct. If it is, awesome! Thank you very much! Here is my fix, although it may not be the cleanest, itworks so far.

//Select number of distinct swimmers
$count_of_swimmers_sql = "SELECT COUNT(DISTINCT SwimmerID) AS CountOfSwimmers 
							FROM d2_2011_cap_times_championship_qualifiers";
$count_of_swimmers_query = mysql_query($count_of_swimmers_sql);
$count_of_swimmers_row = mysql_fetch_assoc($count_of_swimmers_query);
while ($count_of_swimmers_row['CountOfSwimmers'] < 160) {
	//Select events
	$sql = "SELECT EventID
			FROM d2_2011_cap_times_championship_qualifiers
			GROUP BY EventID";
	$query = mysql_query($sql);
	while ($row = mysql_fetch_assoc($query)) {
		//Get current count of distinct swimmers
		$count_of_swimmers_sql = "SELECT COUNT(DISTINCT SwimmerID) AS CountOfSwimmers FROM d2_2011_cap_times_championship_qualifiers";
		$count_of_swimmers_query = mysql_query($count_of_swimmers_sql);
		$count_of_swimmers_row = mysql_fetch_assoc($count_of_swimmers_query);
		if ($count_of_swimmers_row['CountOfSwimmers'] < 160) {
			//Insert 1 swimmer per pass into each event
			$insert_sql = "INSERT INTO d2_2011_cap_times_championship_qualifiers
						 SELECT SwimID, EventID, SwimmerID, Time, TimeStandard, DateSwum
						   FROM d2_2011_cap_times_meet_events
						  WHERE EventID = ".$row['EventID']."
							AND TimeStandard = 'B'
							AND SwimmerID NOT IN (SELECT SwimmerID
													FROM d2_2011_cap_times_championship_qualifiers
												   WHERE EventID = ".$row['EventID'].")
						  ORDER BY Time
						  LIMIT 1";
			$insert_query = mysql_query($insert_sql) or die(mysql_error());	
		} else {
			break;	
		}
	}
	
	//Get current count of distinct swimmers
	$count_of_swimmers_sql = "SELECT COUNT(DISTINCT SwimmerID) AS CountOfSwimmers FROM d2_2011_cap_times_championship_qualifiers";
	$count_of_swimmers_query = mysql_query($count_of_swimmers_sql);
	$count_of_swimmers_row = mysql_fetch_assoc($count_of_swimmers_query);
}

I’m glad you got it sorted out - is there anything else you need? If so, please post your issue and your current code (your zipped file contained your original code from what I could tell…)