Commit Graph

84 Commits

Author SHA1 Message Date
Brion Vibber
532e486a93 Detect when queuedaemon/xmppdaemon parent processes die and kill the child processes.
Keeps stray daemon subprocesses from floating around when we kill the parents via a signal!

Accomplished by opening a bidirectional pipe in the parent process; the children close out the writer end and keep the reader in their open sockets list. When the parent dies, the children see that the socket's been closed out and can perform an orderly shutdown.
2010-03-10 11:54:00 -08:00
Brion Vibber
c74aea589d Stomp queue restructuring for mass scalability:
- Multiplexing queues into groups and for multiple sites.
- Sharing vs breakout configurable per site and per queue via $config['queue']['breakout']
- Detect how many times a message is redelivered, discard if it's killed too many daemons
 - count configurable with $config['queue']['max_retries']
 - can dump the items to files in $config['queue']['dead_letter_dir']

Queue daemon memory & resource leak fixes:
- avoid unnecessary reconnections to memcached server (switch persistent connections back in on second initialization, assuming it's child process)
- monkey-patch for leaky .ini loads in DB_DataObject::databaseStructure() - was leaking 200k per active switch
- applied leak fixes to Status_network as well, using intermediate base Safe_DataObject for both it and Memcache_DataObject

Misc queue fixes:
- correct handling of child processes exiting due to signal termination instead of regular exit
- shutdown instead of infinite respawn loop if we're already past the soft memory limit at startup
- Added --all option for xmppdaemon... still opens one xmpp connection per site that has xmpp active

Cache updates:
- add Cache::increment() method with native support for memcached atomic increment
2010-02-16 09:16:51 -08:00
Brion Vibber
58be61b641 Control channel for queue daemons to request graceful shutdown, restart, or update to listen to a newly added or reconfigured site.
queuectl.php --update -s<site>
  queuectl.php --stop
  queuectl.php --restart

Default control channel is /topic/statusnet-control. For external utilities to send a site update ping direct to the queue server, connect via Stomp and send a message formatted thus:

  update:<nickname>

(Nickname here, *not* server hostname! The rest of the queues will be updated to use nicknames later.)

Note that all currently-connected queue daemons will get these notifications, including both queuedaemon.php and xmppdaemon.php. (XMPP will ignore site update requests for sites that it's not handling.)

Limitations:
* only implemented for stomp queue manager so far
* --update may not yet handle a changed server name properly
* --restart won't reload PHP code files that were already loaded at startup. Still need to stop and restart the daemons from 'outside' when updating code base.
2010-01-26 11:49:49 -08:00
Brion Vibber
0e852def6a XMPP queued output & initial retooling of DB queue manager to support non-Notice objects.
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.

This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues

Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
  This is based on Craig Andrews' work branch to generalize queues to take any
  object, but conservatively leaving out the serialization for now.
  Table updater (preserves any existing queued items) in db/rc3to09.sql

Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
  thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
  directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
  each run through the event loop to watch for memory leaks

To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
  notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
  to queues.
* Find a way to improve interactive performance of the database-backed queue
  handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
  grouping used to split out the XMPP output queue is a bit awkward.
2010-01-21 22:40:35 -08:00
Brion Vibber
0bb23e6fd7 drop debug line from xmppdaemon.php, we're done debugging that 2010-01-21 16:34:26 -08:00
Brion Vibber
71df23642c Extra debug instrumentation for xmppdaemon 2010-01-19 12:26:59 -08:00
Siebrand Mazeland
4af6b7f5c3 Lots of tiny message changes.
* Mostly punctuation updates so that the same message is used consistently in all of StatusNet.
* Some cases of "Title Case" removed, because that does not appear to be used consistently.
2010-01-10 12:26:24 +01:00
Evan Prodromou
d40f0931aa remove dead code for handling direct messages, now done with commands 2009-12-15 10:12:36 -05:00
Ciaran Gultnieks
8109d39a56 Minor typo correction in log message. Seems trivial, unless you are trying to search the log for it. 2009-11-12 09:18:15 +00:00
Evan Prodromou
89ac81c344 remove string-checks from code using Notice::saveNew() 2009-09-29 17:25:52 -04:00
Evan Prodromou
5d09b6b3f0 Merge branch '0.8.x' into 0.9.x
Conflicts:
	EVENTS.txt
	actions/finishremotesubscribe.php
	actions/postnotice.php
	actions/public.php
	actions/remotesubscribe.php
	actions/showstream.php
	actions/updateprofile.php
	actions/userauthorization.php
	classes/laconica.ini
	lib/common.php
	lib/oauthstore.php
	lib/omb.php
2009-08-27 11:16:45 -07:00
Evan Prodromou
4737563b95 a distributed -> the distributed 2009-08-25 18:14:12 -04:00
Evan Prodromou
c8b8f07af1 change Laconica and Control Yourself to StatusNet in PHP files 2009-08-25 18:12:20 -04:00
Evan Prodromou
a94a5fb51a correctly check for max notice length in xmppdaemon 2009-08-21 08:19:09 -04:00
Craig Andrews
3649b9ffbc Set the global $_cur current user so that preferences are loaded.
Thanks singpolyma

http://laconi.ca/trac/ticket/1573
2009-07-21 15:17:54 -04:00
Evan Prodromou
d77a2ee6ad send a ping from the xmppdaemon 2009-07-09 09:31:10 -04:00
Evan Prodromou
eedfaa7d1a clean up payload memory 2009-07-03 23:34:28 -04:00
Evan Prodromou
87e3c52fa8 change name of constructor for xmppdaemon 2009-06-28 17:22:44 -04:00
Evan Prodromou
96814f14db add a lot more logging to xmppdaemon 2009-06-28 17:22:11 -04:00
Evan Prodromou
25c721f6ff if not in daemon mode, xmppdaemon sends log to stdout 2009-06-28 16:38:59 -04:00
Evan Prodromou
cfd2548923 got my background/foreground logic backwards 2009-06-28 16:33:08 -04:00
Evan Prodromou
47e1d2adb8 xmppdaemon.php can stay in foreground 2009-06-28 16:13:08 -04:00
Evan Prodromou
becfd6b3b5 all daemons take an id parameter 2009-06-24 19:31:12 -07:00
Evan Prodromou
875e122a24 xmppdaemon.php uses commandline.inc 2009-06-22 17:07:14 -07:00
Evan Prodromou
0461aafeef Merge branch '0.8.x' into cmdline 2009-06-22 14:51:17 -07:00
Evan Prodromou
aec6456c91 Update copyright dates in files modified in 2009 2009-06-20 16:12:55 -07:00
Evan Prodromou
793a6a1155 change Controlez-Vous to Control Yourself 2009-06-20 16:00:04 -07:00
Evan Prodromou
198afa0a1d change scripts to take server and path from commandline 2009-06-20 14:58:47 -07:00
Adrian Lang
558fa2743e Merge branch '0.7.x' of git://gitorious.org/laconica/bAvatar-clone without the unintended deletion. 2009-04-20 12:15:12 +02:00
Tobias Diekershoff
7196410bb0 shortening links in notices from XMPP
This patch enables shortening of links, that where send from XMPP.
The problem was, that in util.php common_current_user() is not
finding the user account from which is posted, so the service to
shorten is not known, so no shortening at all...

This patch cleans up the xmppdaemon a little bit and hard codes ur1.ca
as shortening service _if_ the user is not set. Ugly but working.
2009-04-18 19:00:20 +02:00
CiaranG
bac3ee95c9 Abort the xmpp-related daemons immediately if xmpp is disabled in the config, otherwise they chew up *lots* of CPU doing nothing 2009-04-16 21:07:59 +01:00
Evan Prodromou
7bcfc9f643 another system message 2009-02-17 12:08:53 -05:00
Evan Prodromou
b264c03d32 move opening brace of class declaration to next line
Another gigantor PEAR coding standards patch. Here, I've moved the
opening curly bracket on a class statement to the following line.

darcs-hash:20081223194923-84dde-77a93de314caadbcb5b70bf346a4648be77a864e.gz
2008-12-23 14:49:23 -05:00
Evan Prodromou
04ef1ba8ee change function headers to K&R style
Another huge change, for PEAR code standards compliance. Function
headers have to be in K&R style (opening brace on its own line),
instead of having the opening brace on the same line as the function
and parameters. So, a little perl magic found all the function
definitions and move the opening brace to the next line (properly
indented... usually).

darcs-hash:20081223193323-84dde-a28e36ecc66672c783c2842d12fc11043c13ab28.gz
2008-12-23 14:33:23 -05:00
Evan Prodromou
eb2f9c98ac replace NULL with null
Another global search-and-replace update. Here, I've replaced the PHP
keyword 'NULL' with its lowercase version. This is another PEAR code
standards change.

darcs-hash:20081223192129-84dde-4a0182e0ec16a01ad88745ad3e08f7cb501aee0b.gz
2008-12-23 14:21:29 -05:00
Evan Prodromou
edbc0c665c replace all tabs with four spaces
The PEAR coding standards decree: no tabs, but indent by four spaces.
I've done a global search-and-replace on all tabs, replacing them by
four spaces. This is a huge change, but it will go a long way to
getting us towards phpcs-compliance. And that means better code
readability, and that means more participation.

darcs-hash:20081223191907-84dde-21e8efe210e6d5d54e935a22d0cee5c7bbfc007d.gz
2008-12-23 14:19:07 -05:00
Evan Prodromou
aa4fb6b71e trim the body of an incoming message
darcs-hash:20081214002239-84dde-9d4d15f37b48ed1830c0423e7e34cb11c7cf9c2d.gz
2008-12-13 19:22:39 -05:00
Evan Prodromou
3c5f1a5b88 xmppdaemon uses 'xml' rather than 'raw' accessor
darcs-hash:20081211183250-5ed1f-73f542de534be2931508841a2c85162e2fe15160.gz
2008-12-11 13:32:50 -05:00
millette
ce08833303 shorten urls for posts > 140 chars only, from anywhere. Only show long urls in title attributes for links we shortened ourselves.
darcs-hash:20081128210114-099f7-4e4cde0a983c2ac6d41efb59b46cb7dbf45dc7a6.gz
2008-11-28 16:01:14 -05:00
Evan Prodromou
0a4905035a new auto-reply regexp (thanks @lilyhill!)
darcs-hash:20081204174754-5ed1f-dcce210332c55a544c35d535c3f3008aa8c165d3.gz
2008-12-04 12:47:54 -05:00
Evan Prodromou
f072147e4e add channels and use command interpreter in different channels
darcs-hash:20081004163213-5ed1f-684ecb464e843b1bbe456c348e56b40a39a83ecd.gz
2008-10-04 12:32:13 -04:00
nick
491af90fa7 XMPP Direct Messaging / Last notice
Direct messaging now works under XMPP, the command syntax is 'd $nick
$message'. Also, a command to fetch a users last notice is in there
('last $nick'). Notification of the recipient of a message is not yet
supported.

darcs-hash:20081003011145-29929-9fce5057066d0d223a0d01c1708df072d91b0e63.gz
2008-10-02 21:11:45 -04:00
CiaranG
dbf80a0f5a Fix error in xmpp help
darcs-hash:20080926214627-f6e2c-f0aa98fe5e76c4aaa3bbaeb400fe85704a31a8c9.gz
2008-09-26 17:46:27 -04:00
Evan Prodromou
ac60342e89 merge CiaranG's changes for subs
darcs-hash:20080922225031-84dde-c6967f46ae642f8943b0de77d9a82892ecadb4ce.gz
2008-09-22 18:50:31 -04:00
Evan Prodromou
eb4df223e6 integrate local changes with Zach's
darcs-hash:20080916210813-84dde-c2cadb63b09f59ac6c76dc1205a56687de9483ce.gz
2008-09-16 17:08:13 -04:00
Evan Prodromou
b7383d5638 fork daemon
darcs-hash:20080626224011-84dde-c6bbfcde6d0802ef3a2346524e0f883e6767c949.gz
2008-06-26 18:40:11 -04:00
Evan Prodromou
b2653f007a remove custom xmppdaemon error handler
darcs-hash:20080830162128-84dde-430fae5f1a0be77db2860c4e6e1c68b961b0944d.gz
2008-08-30 12:21:28 -04:00
CiaranG
bfb22a9932 XMPP daemon updates including help, sub and subsub commands, plus subscribe/unsubcribe logic broken out into standalone module for sharing
darcs-hash:20080816083422-f6e2c-fb8b4e0581719bd8c3f433e661ec9742e1b84cbc.gz
2008-08-16 04:34:22 -04:00
CiaranG
a0b6ebdd2d Bounce oversized XMPP messages back to user (Ticket #112)
darcs-hash:20080814083856-f6e2c-0e56895260fb96dedaa6a663db7e906fee2f7b05.gz
2008-08-14 04:38:56 -04:00
Evan Prodromou
0c71340c9f free and unset DB_DataObjects after we're done with them
darcs-hash:20080906015501-84dde-e787962b9805759224389dd42a211dfa21da3473.gz
2008-09-05 21:55:01 -04:00