Commit Graph

25 Commits

Author SHA1 Message Date
Mikael Nordfeldth
0cd93c2761 Cron plugin added and now default queue handler
Generally the Cron plugin will run if there's still execution time for
1 second since starting the Action processing. If you want to change
this (such as disabling, 0 seconds, or maybe running bigger chunks,
for like 4 seconds) you can do this, where 'n' is time in seconds.

   addPlugin('Cron', array('secs_per_action', n));

Add 'rel_to_pageload'=>false to the array if you want to run the queue
for a certain amount of seconds _despite_ maybe already having run that
long in the previous parts of Action processing.

Perhaps you want to run the cron script remotely, using a machine capable
of background processing (or locally, to avoid running daemon processes),
simply do an HTTP GET request to the route /main/cron of your GNU social.
Setting secs_per_action to 0 in the plugin config will imply that you run
all your queue handling by calling /main/cron (which runs as long as it can).

/main/cron will output "0" if it has finished processing, "1" if it should
be called again to complete processing (because it ran out of time due to
PHP's max_execution_time INI setting).

The Cron plugin also runs events as close to hourly, daily and weekly
as you get, based on the opportunistic method of running whenever a user
visits the site. This means of course that the cron events should be as
fast as possible, not only to avoid delaying page load for users but
also to minimize the risk of running into PHP's max_execution_time. One
suggestion is to only use the events to add new queue items for later processing.

These events are called CronHourly, CronDaily, CronWeekly - however there
is no guarantee that all events will execute, so some kind of failsafe,
transaction-ish method must be implemented in the future.
2013-11-19 14:13:33 +01:00
Mikael Nordfeldth
5308e04e83 Don't disable queuedaemons before Cron stuff is done 2013-11-18 00:50:12 +01:00
Mikael Nordfeldth
1a0e17fea6 Minor queue fixes, probably not very important. Preparing for cron 2013-11-13 18:15:21 +01:00
Jean Baptiste Favre
f175512748 Remove static definition of imdaemon.php as valid daemon. 2013-06-15 18:59:17 +02:00
Craig Andrews
e9995b0f6a Create IM plugin, Pluginize XMPP, Create AIM plugin 2010-01-27 13:48:35 -05:00
Brion Vibber
ec145b73fc Major refactoring of queue handlers to support running multiple sites in one daemon.
Key changes:
* Initialization code moved from common.php to StatusNet class;
  can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
  Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
  the run loop is now managed by IoMaster run via scripts/queuedaemon.php
  IoManager subclasses are woken to handle socket input or polling, and may
  cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
  QueueHandler class; no more need to add a daemon.

The new QueueDaemon runs from scripts/queuedaemon.php:

* This replaces most of the old *handler.php scripts; they've been refactored
  to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
  Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
  (defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon

Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
                           QueueManager [listen or poll] -> QueueHandler
                           XmppManager [ping & keepalive]
                           XmppConfirmManager [poll updates]

Todo:

* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
  handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases

Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-12 20:45:09 -08:00
Zach Copley
78e5a5980a Extract out Facebook app stuff into a plugin 2009-10-20 16:32:30 -07:00
Zach Copley
0fd8e758ad Make queuing and daemons work via events 2009-10-14 04:50:16 +00:00
Evan Prodromou
fe4751de50 add the plugin daemon 2009-09-21 14:44:16 -04:00
Evan Prodromou
f6c70ea327 have to provide full path for daemons 2009-09-21 14:42:20 -04:00
Evan Prodromou
6a088afd4b you can add a daemon to getvaliddaemons 2009-09-21 14:29:43 -04:00
Evan Prodromou
4737563b95 a distributed -> the distributed 2009-08-25 18:14:12 -04:00
Evan Prodromou
c8b8f07af1 change Laconica and Control Yourself to StatusNet in PHP files 2009-08-25 18:12:20 -04:00
Zach Copley
17dcf1c317 Merge branch 'twitter-oauth' into 0.8.x
Conflicts:

	scripts/getvaliddaemons.php
2009-08-10 07:49:51 +00:00
Zach Copley
681bcbf71e Add synctwitterfriends to daemon startup and stop scripts 2009-08-10 07:18:09 +00:00
Jeffery To
93f585446e Added configuration options to enable/disable SMS and Twitter integration.
This disables the IM, SMS and Twitter settings pages and queue handlers
depending on the config options.
2009-08-07 01:18:17 +08:00
CiaranG
db4ffca535 Fix help text for getvaliddaemons.php 2009-07-22 11:21:49 +01:00
Evan Prodromou
5f1b97e2ad no memcached queue handler 2009-06-24 18:02:17 -07:00
Evan Prodromou
06f976f1ac getvaliddaemons.php uses commandline.inc 2009-06-22 16:55:55 -07:00
Evan Prodromou
aec6456c91 Update copyright dates in files modified in 2009 2009-06-20 16:12:55 -07:00
Evan Prodromou
793a6a1155 change Controlez-Vous to Control Yourself 2009-06-20 16:00:04 -07:00
Evan Prodromou
adfb79a9bb Merge branch '0.7.x' into 0.8.x
Conflicts:
	classes/Notice.php
	classes/Profile.php
	lib/common.php
	lib/util.php
	scripts/getvaliddaemons.php
	scripts/stopdaemons.sh
2009-06-08 11:55:32 -07:00
Evan Prodromou
76ee1fd5da Removing inbox and memcached daemon handling 2009-05-30 04:40:47 -04:00
Zach Copley
bc190595d1 Added TwitterStatusFetcher into daemon startup and shutdown subsystem 2009-05-07 02:07:31 -07:00
CiaranG
5b78f95e97 Only start daemons that are required, according to the site config. There is the potential to not start some more - see the checks in getvaliddaemons.php 2009-04-28 13:30:54 +01:00