Commit Graph

598 Commits

Author SHA1 Message Date
Brion Vibber
ab20e75ff8 scripts/strip_geo.php to remove geodata from notices by a given user/profile.
May be slow or run out of memory if run on particularly prolific posters -- not yet optimized for that case.

Note that geodata that has already been sent out to other services (via ostatus, omb, twitter, etc) will not be removed from them.
2010-04-08 16:58:54 -07:00
Brion Vibber
f255b368ed Refactor installer into base class plus web and CLI installers.
scripts/install_cli.php is a bit experimental at present but works.
2010-04-07 17:47:08 -07:00
Brion Vibber
aceaeb99e5 fixup_blocks.php: finds any stray subscriptions in violation of blocks, and removes them. 2010-04-07 11:40:16 -07:00
Evan Prodromou
727ea5a516 Merge branch '0.9.x' into 1.0.x 2010-04-07 10:47:29 -04:00
Christopher Vollick
f09e81ff52 Made it so that settag would list tags.
Very useful to be able to know what tags a site already has.
2010-04-07 07:17:05 -04:00
Siebrand Mazeland
7bcb5c3c9e Follow-up 3ac3bc32fc48508deab171019e3539ece9bd72ff: change po to pot so that exporting from translatewiki.net works.
Signed-off-by: Siebrand Mazeland <s.mazeland@xs4all.nl>
2010-04-07 00:46:03 +02:00
Brion Vibber
5dc169260d Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-04-06 15:22:23 -07:00
Brion Vibber
3ac3bc32fc Some localization cleanup and doc to aid in customization:
* added locale/en/LC_MESSAGES/statusnet.po to make it easier to start customizing English texts
* added notes to locale/README about customizing and how to disable languages you haven't customized
* renamed PO templates from *.po to *.pot to match general conventions and reduce confusion for people trying to find which file they're supposed to edit
2010-04-05 13:19:16 -07:00
Brion Vibber
01a03e34c8 Merge branch '0.9.x' into 1.0.x 2010-03-29 15:15:51 -07:00
Evan Prodromou
f0d905112e Merge branch 'master' into 0.9.x
Conflicts:
	lib/attachmentlist.php
2010-03-28 15:25:02 -04:00
Evan Prodromou
642adc3b00 add an option to run a command as the site owner 2010-03-28 14:38:07 -04:00
Brion Vibber
c3ceaa893f Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 1.0.x 2010-03-24 14:11:01 -07:00
Brion Vibber
ecb009bcf5 Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-03-24 13:57:20 -07:00
Brion Vibber
13d59e0c76 fixup_deletions.php script to look for notices posted by now-deleted profiles and remove them. 2010-03-23 17:24:01 -07:00
Brion Vibber
a5c78a6f1a Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-03-23 08:21:32 -07:00
Zach Copley
b8e97ac709 Some initial media parsing
- Activity now returns a list of activity objects
- Processing of photo objects
2010-03-22 18:55:17 -07:00
Brion Vibber
e89908f261 Merge branch '0.9.x' of git@gitorious.org:statusnet/mainline into 1.0.x
Conflicts:
	lib/channel.php
	scripts/imdaemon.php
2010-03-22 13:56:16 -07:00
Brion Vibber
3e2e88b0df Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-03-18 17:19:28 -07:00
Brion Vibber
0fe82bd0c9 Runtime check for known bad PHP versions with 64-bit stream_select() bug in xmppdaemon.php. 2010-03-18 11:08:27 -07:00
Brion Vibber
67f2f01c5e Merge branch 'testing' into 0.9.x 2010-03-17 12:15:40 -07:00
Brion Vibber
ca5612c451 Merge branch 'master' into testing 2010-03-17 12:15:12 -07:00
Brion Vibber
3e2b806e75 Add scripts/docgen.php to build basic doxygen HTML docs from doc comments, either for core or a given plugin.
Nothing too fancy yet; style and layout needs some loving!
2010-03-15 09:42:25 -07:00
Brion Vibber
6828567ed0 Add forgotten scripts/fixup_files.php to clean up "the h bug" 2010-03-11 11:52:19 -08:00
Brion Vibber
6d73fc1d54 Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-03-10 11:57:38 -08:00
Brion Vibber
54ff17010b Merge branch 'master' of git@gitorious.org:statusnet/mainline into testing 2010-03-10 11:56:51 -08:00
Brion Vibber
532e486a93 Detect when queuedaemon/xmppdaemon parent processes die and kill the child processes.
Keeps stray daemon subprocesses from floating around when we kill the parents via a signal!

Accomplished by opening a bidirectional pipe in the parent process; the children close out the writer end and keep the reader in their open sockets list. When the parent dies, the children see that the socket's been closed out and can perform an orderly shutdown.
2010-03-10 11:54:00 -08:00
Brion Vibber
6ea7e1c06e Merge branch 'testing' into 0.9.x 2010-03-09 14:20:50 -08:00
Brion Vibber
971f1f64f1 Added scripts/command.php, can be used to run commands such as subscription on behalf of users.
This includes whatever support for extended command parsing plugins may have added.

Example:
  ./scripts/command.php -nbrionv sub update@status.net
2010-03-09 13:41:05 -08:00
Brion Vibber
6c4ade4251 Merge branch 'testing' into 0.9.x 2010-03-09 11:27:53 -08:00
Evan Prodromou
61434ebaa2 a script to flush site 2010-03-09 10:55:48 -05:00
Craig Andrews
714d920fae Merge branch '0.9.x' into 1.0.x
Conflicts:
	classes/statusnet.ini
	db/statusnet.sql
	lib/jabber.php
	lib/xmppmanager.php
2010-03-08 17:22:23 -05:00
Evan Prodromou
7faf6ec755 add a script to import Twitter atom feed as notices 2010-03-07 23:42:22 -05:00
Rasmus Lerdorf
086d517b87 Fix a few typos 2010-03-04 23:30:04 -05:00
Brion Vibber
f969d6349c Merge branch 'testing' into 0.9.x
Conflicts:
	db/08to09.sql
2010-03-04 10:16:59 -08:00
James Walker
02f49193d5 adding checkschema to setup script 2010-03-04 11:49:19 -05:00
Brion Vibber
3c4ead4996 Merge branch 'testing' of gitorious.org:statusnet/mainline into 0.9.x
Conflicts:
	db/08to09.sql
	locale/statusnet.po
	scripts/update_po_templates.php
2010-03-01 14:59:26 -08:00
Brion Vibber
00f724d0e6 Localization tweak: include doc comments marked as 'TRANS' in .po file output; these should now get automatically pulled through to the TranslateWiki interface as translator help hints. 2010-03-01 14:10:13 -08:00
Siebrand Mazeland
87c17bc0f0 Also extract _m() used in core.
Signed-off-by: Siebrand Mazeland <s.mazeland@xs4all.nl>
2010-03-01 15:47:43 +01:00
Brion Vibber
9b366547d7 Merge branch 'testing' of gitorious.org:statusnet/mainline into 0.9.x 2010-02-26 12:33:50 -08:00
Evan Prodromou
6781f95c73 Merge branch 'blacklistplus' into testing
Conflicts:
	EVENTS.txt
2010-02-26 12:40:30 -05:00
Brion Vibber
8dfc8f1635 Merge branch 'testing' into 0.9.x 2010-02-26 09:35:28 -08:00
Evan Prodromou
200a32b922 don't choke on dupelimit in createsim.php 2010-02-26 12:19:33 -05:00
Brion Vibber
c8ddcbe0d3 init_conversation.php script to copy old notice conversations into the conversation table 2010-02-25 17:56:01 -08:00
Brion Vibber
4a71753f20 Merge branch 'master' of gitorious.org:statusnet/mainline into testing 2010-02-25 09:28:19 -08:00
Brion Vibber
99c0e16844 Merge commit 'refs/merge-requests/121' of git://gitorious.org/statusnet/mainline into integration 2010-02-25 09:12:45 -08:00
Brion Vibber
66373f8e5a Merge commit 'refs/merge-requests/2220' of git://gitorious.org/statusnet/mainline into integration 2010-02-25 09:11:07 -08:00
Craig Andrews
c187bf5597 Merge branch '0.9.x' into 1.0.x
Conflicts:
	EVENTS.txt
	db/statusnet.sql
	lib/queuemanager.php
2010-02-24 20:52:45 -05:00
Brion Vibber
8914b69d50 Merge branch 'testing' of gitorious.org:statusnet/mainline into 0.9.x 2010-02-24 16:35:57 -08:00
Brion Vibber
3b7084c6dd Fix update_po_templates.php to support the plural and context variants of _m() in plugins 2010-02-24 16:34:45 -08:00
Brion Vibber
85528ccb1f Merge branch 'testing' of gitorious.org:statusnet/mainline into 0.9.x 2010-02-21 11:12:56 -08:00
Christopher Vollick
e717cba19c Add Script To Update Group Avatar URLs
Similar to scripts/updateavatarurl.php.

Works for groups.
Again, I had to do some weird thing because using clone screwed up the find() iteration.
2010-02-18 13:44:16 -05:00
Christopher Vollick
ddef800ec9 Update Avatar URL Did Weird Stuff.
It was only finding the first two avatars and then thinking it was done.

I'm not entirely sure why it was doing that.
I think maybe all the cloning made it forget where it was or something.

Either way, it seems to work now, and really uses less memory.
2010-02-18 12:14:34 -05:00
Brion Vibber
07f145049e Merge branch 'master' into testing 2010-02-17 16:52:13 -08:00
Brion Vibber
ce6be4f836 Queues: redid the breakout control model so we can start up and subscribe to queues without running through the complete site list, which is ok at 1k sites but too slow at 10k.
All breakout queues that we're going to need to listen to now need to be explicitly listed in $config['queue']['breakout'].

Until XMPP is moved to component model, this setting will let the individual processes work with their own queues:
$config['queue']['breakout'][] = 'xmpp/xmppout/' . $config['site']['nickname'];
2010-02-17 16:49:00 -08:00
Craig Andrews
20d6a7caed Merge branch '0.9.x' into 1.0.x
Conflicts:
	lib/queuemanager.php
	lib/xmppmanager.php
	plugins/Xmpp/Fake_XMPP.php
	scripts/imdaemon.php
2010-02-16 13:15:09 -05:00
Brion Vibber
d5cbfe8071 Merge branch 'testing' into 0.9.x
Conflicts:
	lib/iomaster.php
2010-02-16 09:25:09 -08:00
Brion Vibber
81b6b58e33 Merge branch 'master' into testing
Conflicts:
	lib/stompqueuemanager.php
2010-02-16 09:22:02 -08:00
Brion Vibber
c74aea589d Stomp queue restructuring for mass scalability:
- Multiplexing queues into groups and for multiple sites.
- Sharing vs breakout configurable per site and per queue via $config['queue']['breakout']
- Detect how many times a message is redelivered, discard if it's killed too many daemons
 - count configurable with $config['queue']['max_retries']
 - can dump the items to files in $config['queue']['dead_letter_dir']

Queue daemon memory & resource leak fixes:
- avoid unnecessary reconnections to memcached server (switch persistent connections back in on second initialization, assuming it's child process)
- monkey-patch for leaky .ini loads in DB_DataObject::databaseStructure() - was leaking 200k per active switch
- applied leak fixes to Status_network as well, using intermediate base Safe_DataObject for both it and Memcache_DataObject

Misc queue fixes:
- correct handling of child processes exiting due to signal termination instead of regular exit
- shutdown instead of infinite respawn loop if we're already past the soft memory limit at startup
- Added --all option for xmppdaemon... still opens one xmpp connection per site that has xmpp active

Cache updates:
- add Cache::increment() method with native support for memcached atomic increment
2010-02-16 09:16:51 -08:00
Craig Andrews
32084e33a2 Merge branch '0.9.x' into 1.0.x
Conflicts:
	lib/queuemanager.php
2010-02-16 10:25:57 -05:00
Brion Vibber
b2e8d8407c Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-02-08 15:48:52 -08:00
Brion Vibber
4e6f587f86 Merge branch 'master' of git@gitorious.org:statusnet/mainline into testing 2010-02-08 15:47:55 -08:00
Brion Vibber
96ef4435b6 Allow scripts/decache.php to blow out cache for objects that don't exist (anymore).
May miss keys other than the given or primary key, but should work for a lot of common cases where a bad entry's been removed from DB but lingers in cache.
2010-02-08 15:32:20 -08:00
Brion Vibber
c59eee72a8 Merge branch 'testing' of gitorious.org:statusnet/mainline into 0.9.x 2010-02-04 12:23:56 -08:00
Brion Vibber
a020e23086 Merge branch 'master' of gitorious.org:statusnet/mainline into testing 2010-02-04 11:56:07 -08:00
Evan Prodromou
7a7e2162dd Script to update profile URLs 2010-02-03 14:58:29 -05:00
Evan Prodromou
0e0beea5aa clearcache.php column flag was conflicting with default flag 2010-02-03 12:13:20 -05:00
Evan Prodromou
aee73d87ee showcache.php column flag was conflicting with default flag 2010-02-03 12:13:07 -05:00
Evan Prodromou
644a916383 change 'sitetype' to 'siteplan' to use consistent language 2010-02-03 11:55:54 -05:00
Craig Andrews
057ec1fcea Merge branch '0.9.x' into 1.0.x
Conflicts:
	EVENTS.txt
	lib/imqueuehandler.php
	lib/jabber.php
	lib/util.php
	plugins/Xmpp/Sharing_XMPP.php
2010-02-02 17:00:10 -05:00
Brion Vibber
f4c81fa1e1 Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-02-02 09:57:50 -08:00
Brion Vibber
5f14301c55 setconfig.php: list all current settings if no parameters given 2010-02-02 06:15:01 -08:00
Brion Vibber
4f6052d8d8 Apply xopher's fix to add 'sitetype' parameter to setup_status_network.sh, exposed to the email 2010-02-02 05:53:05 -08:00
Brion Vibber
c14ac57b19 Merge branch 'testing' of git@gitorious.org:statusnet/mainline into 0.9.x 2010-02-01 10:30:45 -08:00
Evan Prodromou
5d2c083899 Add a script to clear the cache for a given key
Like showcache.php, clearcache.php clears the cache for a given key.
2010-01-31 22:58:29 -05:00
Evan Prodromou
779204b194 Merge branch 'testing' into 0.9.x
Conflicts:
	actions/apioauthauthorize.php
2010-01-31 15:27:58 -05:00
Evan Prodromou
2a054a50fb live fast, die young in bash scripts 2010-01-29 15:33:35 -05:00
Brion Vibber
288dc3452f Log exceptions from queuedaemon.php if they're not already caught 2010-01-28 22:05:14 -08:00
Brion Vibber
864ce8e276 Fixes for status_network db object .ini and tag setter script 2010-01-28 20:09:17 -08:00
Evan Prodromou
4a0413c027 Add a script to set tags for sites 2010-01-28 20:00:33 -08:00
Craig Andrews
914bc9f9c5 Merge branch '0.9.x' into 1.0.x
Conflicts:
	lib/queuemanager.php
2010-01-27 22:07:04 -05:00
Brion Vibber
fe531dfc6c Shuffle params on setup_status_network; adding fullname and pushing tags up 2010-01-27 16:03:51 -08:00
Brion Vibber
e7a5471287 add additional post-install shell script option for setup_status_network.sh to do any other site-specific setup 2010-01-27 15:37:18 -08:00
Brion Vibber
06cd335897 Add scripts/sendemail.php to send email to a user's address. Updated setup_status_network.sh to create a user with the site's nick, accept site tags, and send a mail to the user (if a template is set)
Email and tag params added to the end:
setup_status_net.sh mysite 'My Site' 'owner@example.com' '1user'

(If multiple tags are needed, separate them with a pipe "|". Be sure to quote properly!)

New parameters for setup.cfg need to be set:
export PHPBASE=/var/www/statusnet
export WILDCARD=example.net
export MAILTEMPLATE=/etc/statusnet/newsite-mail.txt
export MAILSUBJECT="Your new StatusNet site"

$PHPBASE is the base dir for a callable StatusNet install, used to run command-line scripts for user setup.
$WILDCARD is the wildcard domain, needed to build a full server name to pass into command-line scripts.
$MAILTEMPLATE points to a file containing an e-mail message template. '$nickname', '$sitename', and '$userpass' can be used in the template for substitution.
$MAILSUBJECT is the subject line for said email.

To skip sending an email on creation, leave $MAILTEMPLATE blank or point to a non-existing file.
2010-01-27 15:12:19 -08:00
Zach Copley
78079f34e2 Merge branch 'testing' into -1.9.x
* testing: (130 commits)
  HTTP auth provided is evaluated even if it's not required
  Rename rc3to09.sql to rc3torc4.sql to avoid confusion if we add a last-minute change after this!
  Add new oauth tables and modifications to 'consumer' table for rc4
  Centred leaderboard ad
  camelcase the uap param names
  move leaderboard to after the header
  Moved rectangle ad into aside and leaderboard to the right in header.
  Aligning wide skyscraper to the right instead of left
  CSS ids and classes fixed in UAPPlugin
  wrong height for rectangle in BlankAd
  Add the moved BlankAdPlugin
  make BlankAd dir and change to use a 1x1 image
  move BlankAdPlugin to its own dir
  Add BlankAdPlugin to test ad layout in different themes
  make uapplugin an abstract class
  move UAP plugin to core
  Lowercased switch cases in UAP Plugin
  Plugin for Universal Ad Package. Outputs four most widely used ad types.
  Add persistent:true property to Stomp messages so ActiveMQ doesn't decide to discard them even though persistence is enabled on the broker. :) (Thanks Aric!)
  quick fix: use common_path() on realtime update JS so it works with the new JS path code (will pull from main server for now)
  ...

Conflicts:
	actions/apioauthaccesstoken.php
	actions/apioauthauthorize.php
	actions/apioauthrequesttoken.php
	actions/editapplication.php
	actions/newapplication.php
	lib/apiauth.php
	lib/queuemanager.php
	lib/router.php
2010-01-27 14:27:22 -08:00
Craig Andrews
e9995b0f6a Create IM plugin, Pluginize XMPP, Create AIM plugin 2010-01-27 13:48:35 -05:00
Brion Vibber
58be61b641 Control channel for queue daemons to request graceful shutdown, restart, or update to listen to a newly added or reconfigured site.
queuectl.php --update -s<site>
  queuectl.php --stop
  queuectl.php --restart

Default control channel is /topic/statusnet-control. For external utilities to send a site update ping direct to the queue server, connect via Stomp and send a message formatted thus:

  update:<nickname>

(Nickname here, *not* server hostname! The rest of the queues will be updated to use nicknames later.)

Note that all currently-connected queue daemons will get these notifications, including both queuedaemon.php and xmppdaemon.php. (XMPP will ignore site update requests for sites that it's not handling.)

Limitations:
* only implemented for stomp queue manager so far
* --update may not yet handle a changed server name properly
* --restart won't reload PHP code files that were already loaded at startup. Still need to stop and restart the daemons from 'outside' when updating code base.
2010-01-26 11:49:49 -08:00
Brion Vibber
ee4ca8f260 quick fix to console.php: don't save blank lines into readline history 2010-01-25 09:41:40 -08:00
Brion Vibber
055a00bcae drop now-unused --skip-xmpp and --xmpp-only options from queuedaemon.php 2010-01-25 09:36:20 -08:00
Evan Prodromou
c8bc598cfd Merge branch 'testing' into 0.9.x 2010-01-22 14:18:43 -05:00
Evan Prodromou
e666433eb4 Merge branch 'master' into 0.9.x 2010-01-22 14:18:40 -05:00
Brion Vibber
0e852def6a XMPP queued output & initial retooling of DB queue manager to support non-Notice objects.
Queue handlers for XMPP individual & firehose output now send their XML stanzas
to another output queue instead of connecting directly to the chat server. This
lets us have as many general processing threads as we need, while all actual
XMPP input and output go through a single daemon with a single connection open.

This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues

Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob.
  This is based on Craig Andrews' work branch to generalize queues to take any
  object, but conservatively leaving out the serialization for now.
  Table updater (preserves any existing queued items) in db/rc3to09.sql

Code changes to watch out for:
* Queue handlers should now define a handle() method instead of handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning
  thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is queued
  directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage at
  each run through the event loop to watch for memory leaks

To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the actual
  notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice objects
  to queues.
* Find a way to improve interactive performance of the database-backed queue
  handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager. The
  grouping used to split out the XMPP output queue is a bit awkward.
2010-01-21 22:40:35 -08:00
Brion Vibber
0bb23e6fd7 drop debug line from xmppdaemon.php, we're done debugging that 2010-01-21 16:34:26 -08:00
Brion Vibber
71df23642c Extra debug instrumentation for xmppdaemon 2010-01-19 12:26:59 -08:00
Eric Helgeson
e3dc45d103 Merge commit 'refs/merge-requests/117' of git://gitorious.org/statusnet/mainline into integration 2010-01-16 17:42:53 -05:00
Brion Vibber
598072468c --xmpp-only hack for queuedaemon.php to run separate queue daemon with only xmpp threads 2010-01-15 11:13:06 -08:00
Brion Vibber
58bc33850a temporary --skip-xmpp flag on queuedaemon.php, allows to run queue daemons but skip subscription to xmpp-based queues
(still working on making these behave gracefully when server is down)
2010-01-14 15:32:37 -08:00
Brion Vibber
35a2f8de4d actually use the user id instead of 0 ;) 2010-01-13 20:46:44 -08:00
Brion Vibber
82f2fe8cca in case we have cached 'fake' inboxes, make sure we rebuild them running initializeinbox.php 2010-01-13 20:30:42 -08:00
Brion Vibber
ec873922e3 Gracefully skip missing user entries when initializing inboxes in bulk 2010-01-13 20:11:00 -08:00
Evan Prodromou
aa23698553 accept file for initializeinbox.php 2010-01-13 18:28:08 -08:00