Comments on: Web bugs for job scheduling: hack or solution?

By: PHPit - Totally PHP » Creating a “Who’s Online” script with PHP

PHPit - Totally PHP » Creating a “Who’s Online” script with PHP — Sun, 11 Dec 2005 22:48:16 +0000

[…] Our script will store a visitor’s IP address and last active timestamp in a database, and update it every time the visitor requests one of our pages. The script will actually be a web bug, and run completely in the background. If you want to know more about web bugs, have a look at this SitePoint Blog entry by Harry Fuecks, but in a nut shell it’s a small 1×1 image that actually executes PHP, and returns an image. So all our pages will point to a PHP file, like so: […]

By: fryk

fryk — Wed, 30 Nov 2005 11:30:19 +0000

Great article. But it doesn’t work for redirecs. Could anyone know how to make a redirect and continue script execution?

Below is not working. It makes redirect after 3 seconds:

ob_implicit_flush(TRUE); 
@ignore_user_abort(true); 

header("HTTP/1.1 301 Moved Permanently");
header('Content-Length: 0'); 
header('Location: http://example.com'); 
header('Connection: Close'); 

ob_implicit_flush(FALSE); 
ob_start();
		
sleep(3);

// some code

By: Anonymous

Anonymous — Sun, 20 Nov 2005 12:58:28 +0000

I would like to recommend you the most fair and authoritative top 10 web hosting list all over the world.
The to 10 web hosting in November
#1.LunarPages
#2.iPowerWeb
#3.PowWeb
#4.midPhase
#5.Startlogic
#6.Globat
#7.hostony
#8.EasyCGI
#9.dot5
#10.WebsiteSource
See http://www.t10host.com for more information

By: dumky

dumky — Mon, 07 Nov 2005 23:01:49 +0000

Maarten, using PHP as a CLI scripting language for cron jobs does sound like it would help unifying the development environment.

But cron job run in a separate process. That means that any result from the job won’t be directly available in memory for new requests being handled, you have to come up with some kind of inter-process communication solution be it the filesystem or other…
Also, scheduled jobs create an additional deployment requirement.

One specific scenario where I would have needed a good solution for background threads was generating and refreshing a cache of CAPTCHA images. The cache is in-memory to allow more throughput. Using a background thread to generate new images, without saving them to file, allows for a more integrated solution (less boundaries, no need for ACLing directories).

By: Maarten Manders

Maarten Manders — Mon, 07 Nov 2005 18:01:21 +0000

Dumky, you can use cron jobs that execute PHP command line interface (CLI) scripts. It’s a common way to solve those problems.

By: dumky

dumky — Mon, 07 Nov 2005 17:38:22 +0000

I’ve needed to have a background thread running in my web apps a number of times. Why not support this functionality in the web server?

By: Maarten Manders

Maarten Manders — Mon, 07 Nov 2005 08:33:30 +0000

Great article, welcome back Harry!

By: HarryF

HarryF — Fri, 04 Nov 2005 14:01:18 +0000

So we are comparing apples to oranges if what you are after is a system that responds to events rather than set times.

Agreed.

Why wouldn’t the indexer be triggered when the process of saving the updates to the wiki occurs?

Technically what you’re suggesting is doable - in the script that accepts the update, you could hang up the browser in the same way as above then start indexing.

But think the main thing here is whether updates then become the only way the indexes are refreshed. Andi has employed the simplest solution to avoid race conditions, with the rule that only one indexer may run at a time. But what it someone updates a page while the indexer is already running? Then you need some other mechanism to refresh the index later. And you also want to be able to reindex in case of corruption / data loss. Think the web bug approach makes the solution alot simpler.

By: shea

shea — Fri, 04 Nov 2005 12:34:37 +0000

Ahh well the purpose of a cron job, which I’m sure you’re aware of, is to run a specified job at set intervals. So we are comparing apples to oranges if what you are after is a system that responds to events rather than set times. This web bug trick would indeed be best for the latter.

However, I’m not sure I quite get why the indexer needs to be triggered by the web bug trick. Why wouldn’t the indexer be triggered when the process of saving the updates to the wiki occurs?

By: HarryF

HarryF — Fri, 04 Nov 2005 11:16:10 +0000

No, why would you not use cron when you have the option to? Right tool for the job yeah? However, if the question was—would you use this “dodgy hack” if cron wasn’t available? My answer would be a (obviously) resounding yes.

Agreed for general task scheduling but what about the argument that the web bug approach is better integrated with the application than cron can be? If you want something that’s triggered by events happening within your application (such as content being updated via a form), a web bug has a better chance of being able to respond directly the to the event (although it can’t be relied on! - lynx users or those still surfing with images disabled, for example, would be a problem).

In other words think there are a class of problems that can be better solved this way than with cron, such as the Dokuwiki indexer. It’s not a clear distinction but the way Dokuwiki’s indexer works is popular wiki pages have a better chance of getting re-indexed if they change (in fact they’d often be re-indexed immediately after editing, if not other indexer is running). By indexing only one page at a time the overhead is kept reasonably spread. The alternative with cron would likely be something that has to make complete sweeps of all pages and index those that have changed. Each time that job runs it could result is a serious resource hit. Implementing a smarter solution with cron, which spreads the load, would probably turn out more complex than using a web bug.

Should also have mentioned that PHP’s session garbage collector works on a similar basis - it’s incoming requests that fire the garbage collector. If you have no visitors, the session GC won’t run (so expired sessions will still be hanging around).

Also, along the lines of George’s tip regarding images, given an environment you control, if you ran the web bug under a seperate server, like thhptd or even nanoweb, on a subdomain, you’re no longer blocking Apache children.

One other thought (dare I say it?) - this could also work well with AJAX, especially if you need to pass values to the web bug. That also sounds like a legitimate use of AJAX…