wamo/gnu-social
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ec145b73fc
gnu-social/lib
History
Brion Vibber ec145b73fc Major refactoring of queue handlers to support running multiple sites in one daemon. 
Key changes:
* Initialization code moved from common.php to StatusNet class;
  can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
  Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
  the run loop is now managed by IoMaster run via scripts/queuedaemon.php
  IoManager subclasses are woken to handle socket input or polling, and may
  cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
  QueueHandler class; no more need to add a daemon.

The new QueueDaemon runs from scripts/queuedaemon.php:

* This replaces most of the old *handler.php scripts; they've been refactored
  to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
  Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
  (defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon

Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
                           QueueManager [listen or poll] -> QueueHandler
                           XmppManager [ping & keepalive]
                           XmppConfirmManager [poll updates]

Todo:

* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
  handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases

Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-12 20:45:09 -08:00
..
accountsettingsaction.php Added a events for the settings menu items 2009-11-10 17:54:55 -05:00
action.php Fix format specifier on page title 2010-01-11 23:21:09 +00:00
adminform.php Design admin panel mostly done. 2009-11-18 18:25:36 -08:00
adminpanelaction.php let system administrators disallow certain admin panels 2010-01-08 01:00:29 -08:00
api.php Avoid PHP notice when outputting API data for remote users; no $user means no $user->timezone :) 2010-01-12 12:28:24 -08:00
apiauth.php Fix issue with favorited/following always being set to false 2009-12-14 18:16:45 +00:00
apibareauth.php Remove debugging statements 2009-11-04 22:12:01 -08:00
apiprivateauth.php Fix for Ticket #1957 - API methods are not accessible when site is private (0.8->0.9 regression) 2009-11-04 22:03:41 -08:00
arraywrapper.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
attachmentlist.php Revert "* [Cc]an't -> [Cc]annot" 2009-11-08 23:28:51 +01:00
attachmentnoticesection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
attachmenttagcloudsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
authenticationplugin.php Move Authorization and Authentication plugin structures into core, instead of as plugins. 2010-01-05 13:56:22 -05:00
authorizationplugin.php Move Authorization and Authentication plugin structures into core, instead of as plugins. 2010-01-05 13:56:22 -05:00
blockform.php blockform uses profileactionform 2009-11-15 15:57:52 +01:00
cache.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
channel.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
clienterroraction.php move HTTP error code strings to class variables 2009-09-29 17:43:45 -04:00
clientexception.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
columndef.php Fix for auto_increment parameter in auto-created tables via checkschema. 2010-01-04 10:30:58 -08:00
command.php Lots of tiny message changes. 2010-01-10 12:26:24 +01:00
commandinterpreter.php Add repeat command 2009-12-12 21:25:21 -05:00
common.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
connectsettingsaction.php Merge branch '0.9.x' into pluginize-twitter-bridge 2009-09-08 16:02:57 -07:00
currentuserdesignaction.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
curry.php PHP 5.3 closure-based implementation of curry(); old implementation used as fallback for older PHP versions. Added unit tests to confirm they both work! 2009-12-16 09:27:48 -05:00
daemon.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
dberroraction.php Revert "Remove more contractions" 2009-11-09 20:01:46 +01:00
dbqueuemanager.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
default.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
deleteuserform.php add forms for silence, sandbox, delete user 2009-11-15 16:00:29 +01:00
designsettings.php Delete design when user chooses to restore default design, instead 2009-12-03 00:56:53 +00:00
disfavorform.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
error.php Add configuration option to toggle the indenting of the output HTML. Defaults to indent enabled. 2009-12-05 01:03:04 -05:00
event.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
favorform.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
featureduserssection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
feed.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
feedlist.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
form.php Design admin panel mostly done. 2009-11-18 18:25:36 -08:00
galleryaction.php Updated layout for filter by tag form 2009-09-04 15:59:27 +00:00
groupdesignaction.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
groupeditform.php Merge branch '0.8.x' into 0.9.x 2009-08-27 11:16:45 -07:00
grouplist.php Added hAtom for profile and group lists 2009-11-13 12:13:38 +00:00
groupminilist.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
groupnav.php Added hook for the Group navigation items 2009-11-10 13:52:47 +00:00
groupsbymemberssection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
groupsbypostssection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
groupsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
grouptagcloudsection.php Ticket 2141: bugs with weighted popularity lists across year boundary. 2010-01-04 13:01:17 -08:00
htmloutputter.php The structure return by parse_url is an associative array, not an object. 2010-01-06 17:08:01 -05:00
httpclient.php Rebuilt HTTPClient class as an extension of PEAR HTTP_Request2 package, adding redirect handling and convenience functions. 2009-11-02 09:14:15 -08:00
imagefile.php * Mb -> MB (for megabyte) 2009-11-26 23:28:16 +01:00
iomanager.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
iomaster.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
jabber.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
jabberqueuehandler.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
joinform.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
jsonsearchresultslist.php Don't output notices from deleted users. 2010-01-06 13:43:46 -08:00
language.php Ticket 1982: define LC_MESSAGES and friends if PHP didn't predefine them for us. (Known problem on Win32) 2009-12-21 09:06:07 -08:00
leaveform.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
liberalstomp.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
location.php tobyink's location RDF patch 2009-11-19 12:00:25 -05:00
logingroupnav.php Merge branch '0.8.x' into 0.9.x 2009-08-27 11:16:45 -07:00
mail.php Lots of tiny message changes. 2010-01-10 12:26:24 +01:00
mailbox.php Fix regression from 9f372da3da4bd445175eda9155fa7fdd13d3c85e: typo in refactoring caused fatal error on unrecognized message source 2009-11-14 13:10:47 -08:00
mailhandler.php i18n work in the mail handler 2010-01-09 19:04:53 -05:00
mediafile.php Lots of tiny message changes. 2010-01-10 12:26:24 +01:00
messageform.php Updated FormNoticeEnhancements and Counter to use the max text value 2009-12-10 14:16:07 +01:00
microid.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
noticeform.php Removed period 2010-01-11 16:33:46 +01:00
noticelist.php Fixes to bugs where non-local messages were being wrong put in the public timeline and public xmpp feed 2010-01-07 20:59:31 +00:00
noticesection.php Restructure theme.php to define a class Theme 2009-11-08 17:04:46 -05:00
nudgeform.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
oauthclient.php Fix SSL options for Twitter bridge HTTP requests 2009-11-24 19:12:57 +00:00
oauthstore.php change Notice::saveNew() to use named arguments for little-used options 2009-12-11 11:29:51 -05:00
omb.php Ticket 2048: make OMB posting HTTP timeout configurable as $config['omb']['timeout']; defaults to 5 seconds instead of 20-second default in Yadis library 2009-12-01 14:53:27 -08:00
ombqueuehandler.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
ownerdesignaction.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
parallelizingdaemon.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
personalgroupnav.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
personaltagcloudsection.php Ticket 2141: bugs with weighted popularity lists across year boundary. 2010-01-04 13:01:17 -08:00
ping.php Fixes to bugs where non-local messages were being wrong put in the public timeline and public xmpp feed 2010-01-07 20:59:31 +00:00
pingqueuehandler.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
plugin.php add default plugin version information 2010-01-07 17:33:46 -08:00
pluginqueuehandler.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
popularnoticesection.php Ticket 2141: bugs with weighted popularity lists across year boundary. 2010-01-04 13:01:17 -08:00
profileaction.php Include group memberships count & user groups management page link in profile sidebar to match subscribers & subscribees lists. This is the only obvious way to reach the user group membership details page if there are fewer group subs than the max number of icons to show. 2009-11-30 10:41:24 -08:00
profileactionform.php common superclass for actions that operate on a profile and return 2009-11-16 14:28:58 +01:00
profileformaction.php (Puctuation) consistency in clientError() calls. 2009-12-13 18:55:17 +01:00
profilelist.php Added hAtom for profile and group lists 2009-11-13 12:13:38 +00:00
profileminilist.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
profilesection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
publicgroupnav.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
publicqueuehandler.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
queuehandler.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
queuemanager.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
queuemonitor.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
repeatform.php Updated Repeat notice option submit button style 2009-12-23 15:59:31 -05:00
right.php disallow email on faves from sandboxed users 2009-11-16 14:28:55 -05:00
router.php Merge branch 'sessionidparam' into 0.9.x 2010-01-11 00:45:26 -08:00
rssaction.php Add configuration option to toggle the indenting of the output HTML. Defaults to indent enabled. 2009-12-05 01:03:04 -05:00
sandboxform.php add forms for silence, sandbox, delete user 2009-11-15 16:00:29 +01:00
schema.php Support an 'extra' clause when definining a column (e.g.: 'on update 2010-01-05 23:19:13 -08:00
search_engines.php Bringing Sphinx search support up to code: broken out to a plugin, now supports multiple sites on a single server. 2009-11-10 13:44:40 -08:00
searchaction.php i18n for "Search" button text 2009-11-21 16:04:20 +01:00
searchgroupnav.php Ticket 2007: make search tabs grammar forms consistent (People, Notices, Groups) 2009-11-23 06:42:02 -08:00
section.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
servererroraction.php move HTTP error code strings to class variables 2009-09-29 17:43:45 -04:00
serverexception.php Revert "* [Cc]an't -> [Cc]annot" 2009-11-08 23:28:51 +01:00
settingsaction.php Revert "* [Cc]an't -> [Cc]annot" 2009-11-08 23:28:51 +01:00
silenceform.php give SilenceForm proper superclass 2009-11-16 17:05:03 +01:00
smsqueuehandler.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
snapshot.php catch exceptions from snapshot 2010-01-10 14:06:52 -08:00
statusnet.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
stompqueuemanager.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
subgroupnav.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
subpeopletagcloudsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
subs.php Make it impossible to block (and thus unsubscribe from your 2009-12-03 17:06:58 -08:00
subscribeform.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
subscriberspeopleselftagcloudsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
subscriberspeopletagcloudsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
subscriptionlist.php Lots of tiny message changes. 2010-01-10 12:26:24 +01:00
subscriptionspeopleselftagcloudsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
subscriptionspeopletagcloudsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
tabledef.php Break TableDef, ColumnDef classes to separate files so autoloader can find them. 2009-11-24 09:40:47 -08:00
tagcloudsection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
theme.php add a method to Theme class to list available themes 2009-11-08 21:18:57 -05:00
topposterssection.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
unblockform.php Terminology consistency fix: 'Unlock' -> 'Unblock' in unblock form description. 2009-11-18 12:29:47 -08:00
unqueuemanager.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
unsandboxform.php give UnsandboxForm correct superclass 2009-11-16 17:05:31 +01:00
unsilenceform.php give UnsilenceForm correct superclass 2009-11-16 17:05:39 +01:00
unsubscribeform.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
userprofile.php Init UI to show/hide a group of moderation actions. 2009-11-26 18:46:11 +00:00
util.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
webcolor.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
widget.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
xmloutputter.php Add configuration option to toggle the indenting of the output HTML. Defaults to indent enabled. 2009-12-05 01:03:04 -05:00
xmlstringer.php define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
xmppconfirmmanager.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
xmppmanager.php Major refactoring of queue handlers to support running multiple sites in one daemon. 2010-01-12 20:45:09 -08:00
xrdsoutputter.php whitespace adjustments for doxygen 2009-10-29 16:27:23 -04:00