Commit Graph

7566 Commits

Author SHA1 Message Date
Brion Vibber
ec145b73fc Major refactoring of queue handlers to support running multiple sites in one daemon.
Key changes:
* Initialization code moved from common.php to StatusNet class;
  can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
  Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
  the run loop is now managed by IoMaster run via scripts/queuedaemon.php
  IoManager subclasses are woken to handle socket input or polling, and may
  cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
  QueueHandler class; no more need to add a daemon.

The new QueueDaemon runs from scripts/queuedaemon.php:

* This replaces most of the old *handler.php scripts; they've been refactored
  to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
  Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
  (defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon

Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
                           QueueManager [listen or poll] -> QueueHandler
                           XmppManager [ping & keepalive]
                           XmppConfirmManager [poll updates]

Todo:

* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
  handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases

Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-12 20:45:09 -08:00
Brion Vibber
2b10e359fe Avoid PHP notice when outputting API data for remote users; no $user means no $user->timezone :)
Trying to get property of non-object (/srv/com.leuksman.status/lib/api.php:171)
2010-01-12 12:28:24 -08:00
Brion Vibber
4cc9b183d7 Avoid spewing warnings about returning by reference on memcached cache misses by setting a var and returning it instead of trying to return false directly.
This hacky workaround is brought to you by DB_DataObject's PHP 4 roots.
2010-01-12 12:20:45 -08:00
Brion Vibber
9c34d5c107 Actually skip the pingback if XML-RPC extension is missing, instead of considering it then trying anyway and dying of a fatal error. :) 2010-01-12 12:12:31 -08:00
Evan Prodromou
392bc728c7 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-12 12:11:37 -08:00
Sarven Capadisli
21512910c1 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-12 18:47:12 +00:00
Brion Vibber
cbc3c7b141 Ticket 2152: avoid fatal when php.ini disables dl via disabled_functions (function_exists returns false even though it exists and cannot be redefined) 2010-01-12 07:24:43 -08:00
Sarven Capadisli
b9a487a3da Fixed width for form_notice selected view in IE 2010-01-12 03:06:31 +00:00
Evan Prodromou
46b11070fe add inbox to list of always-readwrite tables 2010-01-11 18:31:49 -08:00
Evan Prodromou
ecb3abf84c Merge branch '0.9.x' into inblob 2010-01-11 16:28:27 -08:00
Evan Prodromou
7e5c15fa80 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-11 16:23:48 -08:00
Evan Prodromou
04c76fc4e5 safer storage for diskcacheplugin 2010-01-11 16:23:34 -08:00
Brion Vibber
116c5f6839 dos -> unix line endings on CasAuthentication's CAS library 2010-01-11 15:42:17 -08:00
Siebrand Mazeland
824fd78a8e Localisation updates for !StatusNet from !translatewiki.net !sntrans 2010-01-12 00:36:24 +01:00
Evan Prodromou
bd6571c2e1 Merge branch 'diskcache' into 0.9.x 2010-01-11 15:34:27 -08:00
Zach Copley
91b9c13d15 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-11 23:22:24 +00:00
Zach Copley
3a9e24e077 Fix format specifier on page title 2010-01-11 23:21:09 +00:00
Evan Prodromou
57ac912519 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-11 15:12:13 -08:00
Evan Prodromou
6ef70bf5d9 Merge branch 'sessionidparam' into 0.9.x 2010-01-11 15:12:00 -08:00
Evan Prodromou
3b007934e4 strip out session ID from root URL 2010-01-11 15:10:02 -08:00
Evan Prodromou
5d676352c3 strip out session ID from root URL 2010-01-11 15:09:46 -08:00
Brion Vibber
f397d35527 Regression fix: don't spew notices to log every time we get a non-cached user object 2010-01-11 13:24:52 -08:00
Brion Vibber
be1ac6678d fix long options on deleteuser.php 2010-01-11 13:24:40 -08:00
Sarven Capadisli
60d6d73fca Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-11 16:48:02 +01:00
Sarven Capadisli
c7b768b4c8 Removed period 2010-01-11 16:33:46 +01:00
Sarven Capadisli
70bdaeeb09 Explicitly reseting notice_in-reply-to value 2010-01-11 12:19:53 +00:00
Sarven Capadisli
aaea2b1a96 Apparently, I can't spell my family name. 2010-01-11 13:02:04 +01:00
Evan Prodromou
4afa1ce4f3 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-11 00:45:51 -08:00
Evan Prodromou
8809e577b2 Merge branch 'sessionidparam' into 0.9.x
Conflicts:
	lib/command.php
2010-01-11 00:45:26 -08:00
Evan Prodromou
92deb35bc4 inject session before redirect for openid finish login 2010-01-11 08:40:41 +00:00
Evan Prodromou
5ec25a9691 inject session before redirect for login 2010-01-11 08:40:22 +00:00
Evan Prodromou
ae7469a127 accept session from 2010-01-11 08:39:02 +00:00
root
6f9bdc5b8a Merge branch 'sessionidparam' of http://git.gitorious.org/~evan/statusnet/evans-mainline into sessionidparam 2010-01-11 07:53:15 +00:00
Evan Prodromou
e0eb51e4bb add session ID to local URL when server parts differ 2010-01-10 23:51:57 -08:00
Evan Prodromou
dd7195346c Sever -> server in error message 2010-01-10 22:59:50 -08:00
Evan Prodromou
ad63a9518c Sever -> server in error message 2010-01-10 22:59:32 -08:00
Evan Prodromou
54d532e12f remove redirect to OTP on login from login, register 2010-01-10 22:58:33 -08:00
Siebrand Mazeland
f182d613aa Localisation updates for !StatusNet from !translatewiki.net !sntrans 2010-01-11 01:24:20 +01:00
Evan Prodromou
15b88697a7 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-01-10 14:18:54 -08:00
Evan Prodromou
866dfa6822 Merge branch 'master' into 0.9.x
Conflicts:
	classes/Memcached_DataObject.php
2010-01-10 14:18:19 -08:00
Evan Prodromou
7aa43f3c93 defaultDesign was undefined; fixed that 2010-01-10 14:10:31 -08:00
Evan Prodromou
6fdd52467d catch exceptions from snapshot 2010-01-10 14:06:52 -08:00
Zach Copley
63eddf216f Fix routes for social graph API methods -- this takes care of Ticket #2151 2010-01-10 14:03:10 -08:00
Evan Prodromou
8cf1ef862d catch exceptions from snapshot 2010-01-10 13:54:26 -08:00
Evan Prodromou
2e42d3336a check before saving a thumbnail 2010-01-10 13:25:16 -08:00
Evan Prodromou
f463329b9a check before inserting File_oembed and File_thumbnail 2010-01-10 13:18:53 -08:00
Evan Prodromou
9d3893255a don't put Users with object IDs in the cache, and don't fetch them 2010-01-10 12:31:43 -08:00
Evan Prodromou
30409f7bad debugging code to find passed-in objects in munge_password 2010-01-10 12:07:49 -08:00
Evan Prodromou
91760ae2ae Merge branch 'master' of git@gitorious.org:statusnet/mainline 2010-01-10 11:15:45 -08:00
Siebrand Mazeland
5ad01a7bea Localisation updates for !StatusNet from !translatewiki.net !sntrans 2010-01-10 12:33:03 +01:00