SitePoint Sponsor

User Tag List

Page 2 of 3 FirstFirst 123 LastLast
Results 26 to 50 of 75
  1. #26
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    PHP Code:
    <?
    $logging 
    true;

    //No time limit
    set_time_limit(0);

    //Ignore the browser closing connection
    ignore_user_abort(true);

    while(!
    file_exists('stopfile.txt'))
       {
       
    //Log something - This will see a new line in the logfile every second
       
    db_log(time());
       
    //Anti CPU locking
       
    sleep(1);
       } 

    function 
    db_log($str)
       {
       global
    $logging;         // Is logging enabled
       
    if($logging)
          {
          
    $str date('Y-m-d H:i')."  ".$str."\r\n";

          
    $fh fopen('log.txt''a') or die("can't open file");
          
    fwrite($fh$str);
          
    fclose($fh);
          }
       }
    ?>
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  2. #27
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Tangoforce,
    What will appear in the log file actually? IS just the time is it? Is the event of any error how to log that? Is there any tool to monitor it? Another thing if my server is remotely located how to activate this code?

  3. #28
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Quote Originally Posted by newtomysql View Post
    Dear Tangoforce,
    What will appear in the log file actually?
    With the code i have provided just the time but you can use the db_log() function to log anything you like.

    Quote Originally Posted by newtomysql View Post
    Is the event of any error how to log that?
    Detect it just as you would in any other php code you have written and then just:

    PHP Code:
    db_log($Error); 
    Quote Originally Posted by newtomysql View Post
    Is there any tool to monitor it?
    The log file, or any other function / tool you create. It's a process that you can't see and unlike windows you can't use the task manager to see it running or close it. Therefore you will need to use some sort of output in the loop to see it is running - like the time logging.

    Quote Originally Posted by newtomysql View Post
    Another thing if my server is remotely located how to activate this code?
    Via the browser as i've suggested many times. If you're asking how to install it then you do that via FTP like you would any website.
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  4. #29
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Tangoforce,
    Ok so based on your experience using this method have you use for heavy processing someting like how I anticipate? What is your clue will my problem will adapt well with this browser method?

  5. #30
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I'm unaware of your heavy processing task. All i can say is that in theory this will run more frequently than a cron set to run every two minutes as long as your task takes less than that to complete.

    Unfortunately i can't answer everything, there are somethings you must decide yourself (eg by trying bits of code and seeing how well it works in your scenario).
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  6. #31
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Tangoforce,
    So once I have start your sample code will it go to sleep or keep running for ever and ever? I guess it will sleep for 1 second is it sleep(1)? In the event it crash or go down any tool to alert or restart it ?

  7. #32
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    It MUST sleep for 1 second. This prevents it from a race condition where it will loop as many times as possible in 1 second and locking the CPU causing the machine to freeze. By making it sleep every second it runs one loop per second and for the rest of that second the processor can do other things. It will continue to run like this forever until you stop it.

    You could write another script to check the log file for the last time an entry was made.. OR you could write a timestamp to a database and then have another process check that every second to see if there is a timestamp within the last 2 seconds. If there isn't you have two choices:
    1) The script has crashed - So you restart it automatically using file_get_contents()
    2) Wait another few seconds and check again.

    Ultimately the only tools and methods to control this are the ones that you create and write yourself. If you don't feel confident enough to do this then i highly recommend you look at the cron method again as it will be much simpler.
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  8. #33
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Tangoforce,
    To be honest I am very new to all this method. So I know I got a lot of question to clear my doubts. Is not a matter of confidence but I am kind of confuse too. On the other hand I am also learning more new and useful knowledge. So just kind of lost need to analyse further how to implement all your valuable knowledge.

  9. #34
    Utopia, Inc. silver trophy
    ScallioXTX's Avatar
    Join Date
    Aug 2008
    Location
    The Netherlands
    Posts
    8,907
    Mentioned
    139 Post(s)
    Tagged
    2 Thread(s)
    If you can't follow what the deamon code does I strongly suggest you don't use it but go for the cron job method instead (like tangoforce says in #32).

    What is it you're trying to achieve? Can you describe it in words?
    Rémon - Hosting Advisor

    Minimal Bookmarks Tree
    My Google Chrome extension: browsing bookmarks made easy

  10. #35
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Scallio,
    Let me explain my problem. I have some gps devices which keep pumping gps data into a table. What happens is that those data is raw meaning we dont know whether it is in a geo fence area, any alerts etc. So then I want this service to pickup those raw data and process and store the results in another table so that report generation will be fast and also can send alerts via sms,email etc once detected. So based on this scenarion what is best mechanism. Just to let you know the data inserts are heavy.

  11. #36
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Hmm, thats a difficult one.

    Running a script every second will keep the workload smaller and dealt with on a reguler basis. The downside to this is your inexperience and not quite knowing how to implement the method where many coders would become creative.

    Running a cron is a lot easier for you as a programmer but it will on the other hand leave larger workloads to be processed and may take longer to process them.

    For the time being, until you are more experienced with the principles of input and output and controlling a loop, may i suggest you use the cron job temporarily until you are able to successfully implement my method?
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  12. #37
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Tangoforce,
    What will be the downside with cron? Why do you say larger workload to be processed? I dont quite get it? You mean too many waiting is it?

  13. #38
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    IF you intend (as previously mentioned) running your cron every 2 minutes then the amount of data supplied by your GPS devices may of built up to a large level which could potentially slow things down on your system.

    Cron is only triggered once a minute minimum as far as i'm aware. If you can find a way to make it trigger more frequently then this could be an ideal solution for you instead of my original suggestion.
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  14. #39
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Tangoforce,
    Yes you are right in a minute itself things will be pilling very quickly. So look like your way will the best solution I guess. So in your way when I keep running the select statement to find the unprocessed data it will keep giving me the new lines I guess? So maybe I was thinking lets give the cron job a try first then if things get worse then fall back to your mechnanism. Do you have any other mechanism to solve this problem?

  15. #40
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Unfortunately there isn't really any other mechanism that can do it. It's either Cron, a linux program/daemon or a looping php script as i've shown you.

    Just out of interest, how is the GPS data put into your database? - Is it via a http call?
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  16. #41
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Tango,
    The gps data comes via a java listener. So I am only left with few options let me review and test them and then share with you guys the results.

  17. #42
    Utopia, Inc. silver trophy
    ScallioXTX's Avatar
    Join Date
    Aug 2008
    Location
    The Netherlands
    Posts
    8,907
    Mentioned
    139 Post(s)
    Tagged
    2 Thread(s)
    Can't you pre-process the data at the java listener before you insert it into the database?
    Rémon - Hosting Advisor

    Minimal Bookmarks Tree
    My Google Chrome extension: browsing bookmarks made easy

  18. #43
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Scallio,
    I have tried that but is too complex too many things to process till the whole process of even receiving the data becomes slowdown and miss lots of data. What option do you have?

  19. #44
    Utopia, Inc. silver trophy
    ScallioXTX's Avatar
    Join Date
    Aug 2008
    Location
    The Netherlands
    Posts
    8,907
    Mentioned
    139 Post(s)
    Tagged
    2 Thread(s)
    You could receive the data in java in one thread and have another thread (or multiple threads) for pre-processing the data. Once the data is pre-processed the pre-processor(s) could hand it off to the process that will insert it in the database. That way you have clear separation of concerns and you're keeping all the work in one place (java) instead of spreading it over two (java and php).
    Besides, IMHO Java is a lot more suited for pre-processing stuff like that than PHP is.

    So basically
    receive (1 thread) ===> process (1 or more threads) ===> insert in database (1 thread)
    Rémon - Hosting Advisor

    Minimal Bookmarks Tree
    My Google Chrome extension: browsing bookmarks made easy

  20. #45
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Scallio,
    The problem is that there are many devices. On top of that there are too many things to process e.g poi,geo fence, alerts. So I guess all this will take up time . Wewe have broken into functions in java and tried it too slow response. The problem is that the processing time will become higher and cause delay in inserting data into the database.

  21. #46
    Utopia, Inc. silver trophy
    ScallioXTX's Avatar
    Join Date
    Aug 2008
    Location
    The Netherlands
    Posts
    8,907
    Mentioned
    139 Post(s)
    Tagged
    2 Thread(s)
    Hence the multiple processing threads

    I won't go so far as to state this with 100% certainty, but I'm fairly certain that if Java can't handle it, PHP definately can't handle it.
    Rémon - Hosting Advisor

    Minimal Bookmarks Tree
    My Google Chrome extension: browsing bookmarks made easy

  22. #47
    SitePoint Guru
    Join Date
    Feb 2007
    Posts
    873
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Dear Scallio,
    Here we dont mind the delay as this will only be use for reporting purpose the other one is used for live data. So that is why I do not mind delay this part. Do you know any knowledge of java multiple threads?

  23. #48
    SitePoint Guru
    Join Date
    Aug 2009
    Posts
    669
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    The problem you face though is that if you allow a queue to build up within php then mysql (or any other database) will also start to become locked for pulling data from because there will be an exponential rise in the amount of data waiting to go into it.
    I'll do anything to avoid working on my own code

    Are you using: if (isset($_POST['submit'])) ?
    IE has a bug and does not always send the value.

  24. #49
    Utopia, Inc. silver trophy
    ScallioXTX's Avatar
    Join Date
    Aug 2008
    Location
    The Netherlands
    Posts
    8,907
    Mentioned
    139 Post(s)
    Tagged
    2 Thread(s)
    Quote Originally Posted by newtomysql View Post
    Do you know any knowledge of java multiple threads?
    Yup, it gets messy real fast
    No it's doable as long as you don't need concurrency, that's pretty nasty. However explaining how concurrency works is too much to explain in a post on a forum (and TBH my Java is starting to get a bit rusty) so I suggest that of you're interested you google it and start a new thread in the Java forum if anything isn't clear.

    When is the data coming in? Is it all the time, 24/7?
    If there are breaks in the incoming data (like say there is no data at night), you might want to create the reports then. If that's not possible and you do run everything at the same time you'll still clog up the database like tangoforce just said.
    It sounds like your handling a whole lot of data here so if I were you I'd really sit down and think about how to handle it, and what to handle when. This is not something you can just try and hope the best of it.
    Rémon - Hosting Advisor

    Minimal Bookmarks Tree
    My Google Chrome extension: browsing bookmarks made easy

  25. #50
    SitePoint Zealot Dorsey's Avatar
    Join Date
    Feb 2004
    Location
    NJ
    Posts
    103
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    I'm sure that someone else has spotted that doing something and then sleeping for one second is not the same as executing every second. You do need cron for that, and something like a semaphore to prevent overlap.

    Also, keep in mind that *nix is NOT a real-time system. When I wrote real-time systems for the telephone company and laboratory automation systems for the pharmaceutical industry, we used true real-time systems (HP and DEC at the time). The first thing the process did was re-queue itself and then do the work; cron is close enough to that for what you want to do.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •