I have a PHP program that opens a text data file, reads it, adds to it, and saves it again.
The question is: how big can that file get, or how many lines can it contain, before PHP runs out of space or memory?
Before I start getting error codes I’d like to have some idea of what the limits are, or how to find out - and how to change them.
Thank you;
Ian
First are you reading the entire file or just the last line?
If you are only reading the last line in the file, then PHP memory limit shouldn’t be much of a problem, however, I would strongly recommend doing a prepend (using the ‘x’ instead of ‘w’) to the file so you consistently read the first line instead of the last, and then you insert a new first line when writing (makes coding easier and less work).
Your limits will be the hard drive storage space and the php max memory limit (if you read the entire file).
Unfortunately, the nature of the file means I have to read the entire file, change the first line, add additional lines to the end, then save it.
This should be all server-side, I think.
Will the capacity of the variable that the file is ‘read into’ be a problem? I’m still used to VB where the variable size had to be defined - but that does date back to a time when a big computer had 4mb of RAM!
Since you will be reading the file into a variable, your only limit will be the max memory limit in your php.ini file.
Thank you for that info. I should have some breathing room. Now I have to find the php.ini file.
This isn’t running on “my” server. My pages are hosted on a very large server system, somewhere in Arizona.
I have looked through my options, and there appears to be no way I can change the individual settings for the PHP extensions - which makes sense because it’s probably a shared server.
If you run
<?php echo phpinfo(); ?>
You can look through the output and find the max memory limit. Remove that line/file immediately after finding what you need though, as it gives out a lot of information you won’t want to make public.
Yes, I was just looking through that readout, and apparently I have 64m to work with - which should be sufficient.
Thank you for your help.
Maybe this idea is redundant now you don’t think memory will be an issue.
This advice about using sed might have been useful IF on a *nix machine AND exec() had not been disabled (check phpinfo() again). The search on [google]using php exec and sed[/google] seems to suggest that could be a bit more efficient.
I have to say I have not tested this, merely sharing an idea.
If your file consists of separate lines then think whether you will be able to get away with reading each line separately and dealing only with one line at a time (in a loop). This way you will not have to load the whole file into a variable, you will only need to use fgets() to read a single line so then you need enough memory to hold just one line. Of course, this will mean not adding the result to any string or array - just loading each new line to the same variable.
Adding additional lines is not a problem at all if you open the file in a or a+ mode.
Well, like I said, I have to read - and then change the first line, then add one or more lines to the end of the file. If it was only a matter of simple adding to the file I imagine there must be some sort of Append function.
If the file becomes larger than 64M I may have to rethink the entire process. Ugh!
Okay, if you have to change the first line and the line may have to be of different length then yes, simply appending will not do. But do you really have to read the whole file into a variable instead of reading it line by line and performing all your required logic in this way? If you can read it line by line then there is a different solution that should work: read the original file line by line and instead of creating a whole new output in a variable open another (temporary) file for writing and write the changed data to the new file line by line as you are reading from the original. If linear reading is not enough for you then you can use fseek() to start reading from any point in the file. As you are done then simply rename the temporary file to the original which will replace the original file with the new file. This will also require very little memory provided the lines are not very long.
Yes, that looks like the best solution. Use a simple loop to read each line, modify as necessary, and save to a temp file, then rename back.
Thank you for the idea.
Sometimes it’s easy to get fixated on a single method of achieving the desired results. A second perspective is always helpful.