Commit Graph

51 Commits

Author SHA1 Message Date
Mikael Nordfeldth
2f86cd8602 utf8mb4 conversion on database with index adjusts 2015-02-12 18:18:55 +01:00
Mikael Nordfeldth
f7548ed7e9 cacheKey change for Status_network class
From statusnet to gnusocial.
2015-02-05 12:06:01 +01:00
Mikael Nordfeldth
2dfa0bfcee function delete in dataobjects now don't break strict syntax 2013-10-29 10:20:57 +01:00
Mikael Nordfeldth
3ba6374b9d Memcached_DataObject extensions got their update functions more consistent 2013-10-28 19:36:05 +01:00
Mikael Nordfeldth
202f6ad7a9 Removing legacy code and fixup_* for Status_network tags 2013-10-15 10:54:03 +02:00
Mikael Nordfeldth
b3e61ce7d0 Stronger typing, require array where param array 2013-08-29 10:27:39 +02:00
Mikael Nordfeldth
2a4dc77a63 The overloaded DB_DataObject function staticGet is now called getKV
I used this hacky sed-command (run it from your GNU Social root, or change the first grep's path to where it actually lies) to do a rough fix on all ::staticGet calls and rename them to ::getKV

   sed -i -s -e '/DataObject::staticGet/I!s/::staticGet/::getKV/Ig' $(grep -R ::staticGet `pwd`/* | grep -v -e '^extlib' | grep -v DataObject:: |grep -v "function staticGet"|cut -d: -f1 |sort |uniq)

If you're applying this, remember to change the Managed_DataObject and Memcached_DataObject function definitions of staticGet to getKV!

This might of course take some getting used to, or modification fo StatusNet plugins, but the result is that all the static calls (to staticGet) are now properly made without breaking PHP Strict Standards. Standards are there to be followed (and they caused some very bad confusion when used with get_called_class)

Reasonably any plugin or code that tests for the definition of 'GNUSOCIAL' or similar will take this change into consideration.
2013-08-18 13:13:56 +02:00
Mikael Nordfeldth
e95f77d34c Updating all Memcached_DataObject extended classes to Managed_DataObject
In some brief tests, this causes no problems.

In this state however, you would need to modify DB_DataObject to have a static declaration of staticget (and probably pkeyGet). The next commit will change the staticGet overload to a unique function name (like getKV for getKeyValue), which means we can properly call the function by PHP Strict Standards.
2013-08-18 12:32:32 +02:00
Evan Prodromou
b6df936454 Better handling for combined memcache identifiers 2011-06-10 19:13:33 -04:00
Evan Prodromou
3dbfa97979 hide errors when deleting cached status_network_tag keys 2011-06-10 18:57:17 -04:00
Evan Prodromou
08cf6827a9 try to make it so Status_network_tag can go fingerpoken in Status_network's static vars 2011-06-10 18:12:51 -04:00
Evan Prodromou
7ed3b9cf3f Status_network::setupDB() sets up related tables too 2011-04-28 15:29:36 -07:00
Siebrand Mazeland
9587f9f55b * i18n/L10n and translator documentation updates.
* whitespace and indentation updates
2010-09-28 23:42:18 +02:00
James Walker
444959a789 Status_network::encache() doesn't exist 2010-09-17 16:33:02 -04:00
Brion Vibber
63fd2332f0 Use Status_network's caching settings when loading Status_network_tag entries; cache entries are batched per site and will be cleared when tags are inserted/deleted using the main interface.
(with fixes from tagcache branch)
2010-08-25 12:22:42 -07:00
Siebrand Mazeland
125ff142e8 * mark a few message for translation
* add translator documentation
2010-07-29 13:36:08 +02:00
James Walker
5688c635a6 backwards compatibility for old tags format in hasTag 2010-07-28 12:13:53 -04:00
James Walker
29b8a6a18f don't try to save empty tags 2010-07-28 11:57:54 -04:00
James Walker
7065450f03 normalizing tags for status_network 2010-07-20 17:34:58 -07:00
James Walker
b667092153 adding a Status_network::updateKeys() method, since DB_DataObject update doesn't do keys. 2010-06-17 13:44:17 -04:00
Brion Vibber
8b9436e8ae Option to divert PuSH items directly to the target site's queue when local 2010-06-03 18:06:55 -07:00
Brion Vibber
c74aea589d Stomp queue restructuring for mass scalability:
- Multiplexing queues into groups and for multiple sites.
- Sharing vs breakout configurable per site and per queue via $config['queue']['breakout']
- Detect how many times a message is redelivered, discard if it's killed too many daemons
 - count configurable with $config['queue']['max_retries']
 - can dump the items to files in $config['queue']['dead_letter_dir']

Queue daemon memory & resource leak fixes:
- avoid unnecessary reconnections to memcached server (switch persistent connections back in on second initialization, assuming it's child process)
- monkey-patch for leaky .ini loads in DB_DataObject::databaseStructure() - was leaking 200k per active switch
- applied leak fixes to Status_network as well, using intermediate base Safe_DataObject for both it and Memcache_DataObject

Misc queue fixes:
- correct handling of child processes exiting due to signal termination instead of regular exit
- shutdown instead of infinite respawn loop if we're already past the soft memory limit at startup
- Added --all option for xmppdaemon... still opens one xmpp connection per site that has xmpp active

Cache updates:
- add Cache::increment() method with native support for memcached atomic increment
2010-02-16 09:16:51 -08:00
Brion Vibber
58be61b641 Control channel for queue daemons to request graceful shutdown, restart, or update to listen to a newly added or reconfigured site.
queuectl.php --update -s<site>
  queuectl.php --stop
  queuectl.php --restart

Default control channel is /topic/statusnet-control. For external utilities to send a site update ping direct to the queue server, connect via Stomp and send a message formatted thus:

  update:<nickname>

(Nickname here, *not* server hostname! The rest of the queues will be updated to use nicknames later.)

Note that all currently-connected queue daemons will get these notifications, including both queuedaemon.php and xmppdaemon.php. (XMPP will ignore site update requests for sites that it's not handling.)

Limitations:
* only implemented for stomp queue manager so far
* --update may not yet handle a changed server name properly
* --restart won't reload PHP code files that were already loaded at startup. Still need to stop and restart the daemons from 'outside' when updating code base.
2010-01-26 11:49:49 -08:00
Brion Vibber
ad6f0501ff Site metadata tags in status_network: single 'tags' field, pipe-separated.
$sn->tags() returns tag list as array; $sn->hasTag('blah') to check for a particular tag only

Could be used to control things in config file:

  $sn = Status_network::setupSite($_server, $_path, $_wildcard);
  if (!$sn) { die("No such site"); }
  if ($sn->hasTag('individual')) { /* blah */ }

Note memcached keys are unchanged; if tags are changed from an external tool clear:
  statusnet:<dbname>:status_network:<key>:<val>
  for <key>s 'nickname', 'hostname', and 'pathname'
2010-01-26 10:33:20 -08:00
Evan Prodromou
238998eca0 save nickname and wildcard when setting up status network 2010-01-24 18:18:24 -05:00
Brion Vibber
ec145b73fc Major refactoring of queue handlers to support running multiple sites in one daemon.
Key changes:
* Initialization code moved from common.php to StatusNet class;
  can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
  Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
  the run loop is now managed by IoMaster run via scripts/queuedaemon.php
  IoManager subclasses are woken to handle socket input or polling, and may
  cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
  QueueHandler class; no more need to add a daemon.

The new QueueDaemon runs from scripts/queuedaemon.php:

* This replaces most of the old *handler.php scripts; they've been refactored
  to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
  Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
  (defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon

Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
                           QueueManager [listen or poll] -> QueueHandler
                           XmppManager [ping & keepalive]
                           XmppConfirmManager [poll updates]

Todo:

* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
  handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases

Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work
2010-01-12 20:45:09 -08:00
Evan Prodromou
e2dee5fedb always set site/server to hostname if it exists 2010-01-10 00:20:08 -08:00
Evan Prodromou
3d723ed1ed allow hostname with SSL 2010-01-09 22:49:26 -08:00
Evan Prodromou
e8d85a1ef5 use nickname, not sitename, in domain for SSL 2010-01-09 22:48:05 -08:00
Evan Prodromou
deb5ee6154 correct superglobal variable name 2010-01-09 16:31:25 -08:00
Evan Prodromou
5ca41b6870 redirect to sitename.wildcard for SSL 2010-01-09 16:19:45 -08:00
Brion Vibber
53c86c43c4 Bringing Sphinx search support up to code: broken out to a plugin, now supports multiple sites on a single server.
Upgrade notes:
* Index names have changed from hardcoded 'Identica_people' and 'Identica_notices' to use the database name and actual table names. Must reindex.

New events:
* GetSearchEngine to override default search engine class selection from plugins

New scripts:
* gen_config.php generates a sphinx.conf from database configuration (with theoretical support for status_network table, but it doesn't seem to be cleanly queriable right now without knowing the db setup info for that. Needs generalized support.)
* Replaced old sphinx-indexer.sh and sphinx-cron.sh with index_update.php

Other fixes:
* sphinx.conf.sample better matches our live config, skipping unused stopword list and using a more realistic indexer memory limit

Further notes:
* Probably doesn't work right with PostgreSQL yet; Sphinx can pull from PG but the extraction queries currently look like they use some MySQL-specific functions.
2009-11-10 13:44:40 -08:00
Evan Prodromou
c628029ef1 Status_network had wrong ini file 2009-08-28 10:42:34 -07:00
Evan Prodromou
df86aa7214 define LACONICA and accept LACONICA for backwards compatibility 2009-08-26 10:41:36 -04:00
Evan Prodromou
3567b9d708 global search and replace for laconica -> statusnet 2009-08-25 18:53:24 -04:00
Evan Prodromou
865b716f09 change LACONICA to STATUSNET 2009-08-25 18:42:34 -04:00
Evan Prodromou
4737563b95 a distributed -> the distributed 2009-08-25 18:14:12 -04:00
Evan Prodromou
c8b8f07af1 change Laconica and Control Yourself to StatusNet in PHP files 2009-08-25 18:12:20 -04:00
Evan Prodromou
166e4a4b58 bad string compare in Status_network 2009-06-28 14:48:19 -04:00
Evan Prodromou
d4f1637f8a fallback for www. addresses 2009-06-28 14:45:33 -04:00
Evan Prodromou
39117bf88c forgot to disinherit Memcached_DataObject in Status_network 2009-06-15 21:55:56 -07:00
Evan Prodromou
0a886868fb Add some basic memcached handling to status_network
Status_network can't be a subclass of Memcached_DataObject -- the
latter is too entrenched in Laconica's memc handling functions, which
aren't loaded when Status_network is running! But the importance of
caching these values can't be overstated. So, a considerably
slimmed-down version of the Memcached_DataObject code is transcribed
into Status_network.
2009-06-15 21:33:56 -07:00
Evan Prodromou
8c24a3bc92 a little better query handling in redirect code 2009-06-15 16:55:01 -07:00
Evan Prodromou
f809663245 a little better query handling in redirect code 2009-06-15 16:51:49 -07:00
Evan Prodromou
2f82a3d44c forgot some functions aren't available at status time 2009-06-15 16:43:39 -07:00
Evan Prodromou
587b7a8b2a redirect on non-canonical server name 2009-06-15 16:40:53 -07:00
Evan Prodromou
6532f0dff6 strncmp -> strcasecmp 2009-06-15 14:29:25 -07:00
Evan Prodromou
d6ff702d7f Return network from network setup function
Return the network from the network setup function. Also, special-case
for when we get a server name the same as the wildcard.
2009-06-15 13:34:52 -07:00
Evan Prodromou
eb6a60ef88 updates to Status_network 2009-06-15 08:54:52 -07:00
Evan Prodromou
68d90bcab0 some class files had x bit set 2009-05-22 21:11:46 -04:00