Commit Graph

32 Commits

Author SHA1 Message Date
Brion Vibber
0e852def6a XMPP queued output & initial retooling of DB queue manager to support non-Notice objects.
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.

This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues

Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
  This is based on Craig Andrews' work branch to generalize queues to take any
  object, but conservatively leaving out the serialization for now.
  Table updater (preserves any existing queued items) in db/rc3to09.sql

Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
  thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
  directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
  each run through the event loop to watch for memory leaks

To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
  notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
  to queues.
* Find a way to improve interactive performance of the database-backed queue
  handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
  grouping used to split out the XMPP output queue is a bit awkward.
2010-01-21 22:40:35 -08:00
Brion Vibber
ec145b73fc Major refactoring of queue handlers to support running multiple sites in one daemon.
Key changes:
* Initialization code moved from common.php to StatusNet class;
  can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
  Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
  the run loop is now managed by IoMaster run via scripts/queuedaemon.php
  IoManager subclasses are woken to handle socket input or polling, and may
  cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
  QueueHandler class; no more need to add a daemon.

The new QueueDaemon runs from scripts/queuedaemon.php:

* This replaces most of the old *handler.php scripts; they've been refactored
  to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
  Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
  (defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon

Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
                           QueueManager [listen or poll] -> QueueHandler
                           XmppManager [ping & keepalive]
                           XmppConfirmManager [poll updates]

Todo:

* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
  handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases

Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-12 20:45:09 -08:00
Zach Copley
20af83d316 Add version info for Facebook, TwitterBridge and RSSCloud plugins 2010-01-08 00:09:23 -08:00
Zach Copley
e6c8f6a8f8 Removed crazy redundant broadcasting of notices by the FB app 2009-12-31 22:53:46 +00:00
Zach Copley
962eed904c - Use a stripped down new notice form for FB app because FB canvas apps
can't support image upload via multipart/form-data (and location sharing
  is iffy).
- Deal with new error code 100 from Facebook, which seem to be for
  inactive accounts.
2009-12-31 22:39:01 +00:00
Zach Copley
5621f85835 Change inline CSS stylesheet to be on a single line so it doens't blow out syntax highlighting in my editor 2009-12-31 22:39:01 +00:00
Zach Copley
450cd6774a Update to external Facebook libs 2009-12-31 22:39:01 +00:00
Evan Prodromou
79f81ad76d change Notice::saveNew() to use named arguments for little-used options 2009-12-11 11:29:51 -05:00
Brion Vibber
4b5e977a7b New _m() gettext wrapper with smart detection of plugin domains. Plugin base class registers your gettext files if present at initialization.
update_pot.sh replaced with update_po_templates.php which can do core, plugins, or all (default).
Top-level Makefile added to build .mo files for plugins as well as core.

As described on list:
http://lists.status.net/pipermail/statusnet-dev/2009-December/002869.html
2009-12-08 12:17:11 -08:00
Craig Andrews
51f0dd5e37 Add configuration option to toggle the indenting of the output HTML. Defaults to indent enabled. 2009-12-05 01:03:04 -05:00
Craig Andrews
4c8bed8ba0 Use inlineScript() everywhere inline scripts are written 2009-12-04 19:41:51 -05:00
Zach Copley
e6842b7189 Fixed bug where reply-sync bit wasn't getting saved 2009-11-25 01:36:17 +00:00
Zach Copley
b586063ebe Forgot to render the nav menu when on FB Connect login tab 2009-11-25 01:11:29 +00:00
Zach Copley
f12972a50f Facebook plugin no longer takes over Login and Connect settings nav menus 2009-11-25 00:43:20 +00:00
Zach Copley
4bace8f1a5 Add routing rules for FB app in front of other rules 2009-11-20 11:13:24 +00:00
Sarven Capadisli
1bac324072 Moved class="pagination" to child element and removed element 2009-11-13 12:26:24 +00:00
Brion Vibber
088081675f Revert "Remove more contractions"
This reverts commit 5ab709b739.

Missed this one yesterday...
2009-11-09 20:01:46 +01:00
Siebrand Mazeland
b10f362ede Merge branch '0.9.x' of git://gitorious.org/statusnet/mainline into 0.9.x 2009-11-08 23:33:58 +01:00
Siebrand Mazeland
5ab709b739 Remove more contractions
* doesn't
* won't
* isn't
* don't
2009-11-08 23:32:15 +01:00
Brion Vibber
fc5002015b Revert "* [Cc]an't -> [Cc]annot"
This reverts commit 0ab17f382b.
2009-11-08 23:28:51 +01:00
Siebrand Mazeland
0ab17f382b * [Cc]an't -> [Cc]annot
* [Cc]ould't -> [Cc]ould not
2009-11-08 23:22:38 +01:00
Brion Vibber
fd650715fa Warning cleanup: drop reference on router parameter to RouterInitialized event handlers.
We don't (and don't need to) pass a reference here, and the mix can trigger warnings.
2009-10-29 13:55:37 -04:00
Zach Copley
a1798039b2 Specify the number of cols for the Facebook app's multi-friend inviter
widget so it fits better into Facebook's new layout.
2009-10-27 21:07:17 +00:00
Zach Copley
4de05f1e47 Forgot to move the facebookapp.css file into the new Facebook plugin 2009-10-27 21:00:26 +00:00
Zach Copley
6d6de3c1c7 Move Twitter and Facebook-specific mail notifications to their respective plugins 2009-10-23 23:38:45 +00:00
Zach Copley
62ba25f53a Some phpcs cleanup 2009-10-23 23:05:23 +00:00
Zach Copley
ec92cab6ff Removed Facebook stuff from the StatusNet README and wrote a new README
for the Facebook plugin.
2009-10-21 17:53:23 -07:00
Zach Copley
6a2185a26b Make paths and class loading work right 2009-10-21 22:02:10 +00:00
Zach Copley
3e5c8f28c3 Fix references 2009-10-21 17:29:54 +00:00
Zach Copley
aa58e8a738 Combine Facebook and Facebook Connect plugins into one big plugin 2009-10-21 01:17:31 +00:00
Zach Copley
d07c9d8750 Gather all the Facebook stuff together in one place 2009-10-21 01:07:03 +00:00
Zach Copley
78e5a5980a Extract out Facebook app stuff into a plugin 2009-10-20 16:32:30 -07:00