Commit Graph

20 Commits

Author SHA1 Message Date
Brion Vibber
ec145b73fc Major refactoring of queue handlers to support running multiple sites in one daemon.
Key changes:
* Initialization code moved from common.php to StatusNet class;
  can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
  Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
  the run loop is now managed by IoMaster run via scripts/queuedaemon.php
  IoManager subclasses are woken to handle socket input or polling, and may
  cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
  QueueHandler class; no more need to add a daemon.

The new QueueDaemon runs from scripts/queuedaemon.php:

* This replaces most of the old *handler.php scripts; they've been refactored
  to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
  Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
  (defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon

Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
                           QueueManager [listen or poll] -> QueueHandler
                           XmppManager [ping & keepalive]
                           XmppConfirmManager [poll updates]

Todo:

* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
  handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases

Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-12 20:45:09 -08:00
Zach Copley
78e5a5980a Extract out Facebook app stuff into a plugin 2009-10-20 16:32:30 -07:00
Zach Copley
0fd8e758ad Make queuing and daemons work via events 2009-10-14 04:50:16 +00:00
Evan Prodromou
fe4751de50 add the plugin daemon 2009-09-21 14:44:16 -04:00
Evan Prodromou
f6c70ea327 have to provide full path for daemons 2009-09-21 14:42:20 -04:00
Evan Prodromou
6a088afd4b you can add a daemon to getvaliddaemons 2009-09-21 14:29:43 -04:00
Evan Prodromou
4737563b95 a distributed -> the distributed 2009-08-25 18:14:12 -04:00
Evan Prodromou
c8b8f07af1 change Laconica and Control Yourself to StatusNet in PHP files 2009-08-25 18:12:20 -04:00
Zach Copley
17dcf1c317 Merge branch 'twitter-oauth' into 0.8.x
Conflicts:

	scripts/getvaliddaemons.php
2009-08-10 07:49:51 +00:00
Zach Copley
681bcbf71e Add synctwitterfriends to daemon startup and stop scripts 2009-08-10 07:18:09 +00:00
Jeffery To
93f585446e Added configuration options to enable/disable SMS and Twitter integration.
This disables the IM, SMS and Twitter settings pages and queue handlers
depending on the config options.
2009-08-07 01:18:17 +08:00
CiaranG
db4ffca535 Fix help text for getvaliddaemons.php 2009-07-22 11:21:49 +01:00
Evan Prodromou
5f1b97e2ad no memcached queue handler 2009-06-24 18:02:17 -07:00
Evan Prodromou
06f976f1ac getvaliddaemons.php uses commandline.inc 2009-06-22 16:55:55 -07:00
Evan Prodromou
aec6456c91 Update copyright dates in files modified in 2009 2009-06-20 16:12:55 -07:00
Evan Prodromou
793a6a1155 change Controlez-Vous to Control Yourself 2009-06-20 16:00:04 -07:00
Evan Prodromou
adfb79a9bb Merge branch '0.7.x' into 0.8.x
Conflicts:
	classes/Notice.php
	classes/Profile.php
	lib/common.php
	lib/util.php
	scripts/getvaliddaemons.php
	scripts/stopdaemons.sh
2009-06-08 11:55:32 -07:00
Evan Prodromou
76ee1fd5da Removing inbox and memcached daemon handling 2009-05-30 04:40:47 -04:00
Zach Copley
bc190595d1 Added TwitterStatusFetcher into daemon startup and shutdown subsystem 2009-05-07 02:07:31 -07:00
CiaranG
5b78f95e97 Only start daemons that are required, according to the site config. There is the potential to not start some more - see the checks in getvaliddaemons.php 2009-04-28 13:30:54 +01:00