wamo/gnu-social
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,851 Commits 8 Branches 0 Tags 330 MiB
4ba7c4a021
Commit Graph

78 Commits

Author SHA1 Message Date
Mikael Nordfeldth
3bc2454e91 Avoid excessive remote requests on oEmbed lookups 
I noticed that each time a notice was accessed it'd do a remote lookup
with HEAD (and continue despite 404 etc.) and then another attempt to
download the resource. If this wasn't successful new attempts would be
made for each loading of the resource, which is extremely resource
intensive.

Whenever we can say "it's been n seconds since the last attempt" we
could probably enable this again - or just manually reload remote
thumbnails (as part of the StoreRemoteMedia plugin etc.)
 2017-07-10 14:41:03 +02:00
Mikael Nordfeldth
06b25f384a File_redirection->getFile could never get the file anyway if $redir->file_id was empty... 2017-05-02 09:07:00 +02:00
Mikael Nordfeldth
f6d4d00e02 I think this will stop my daemons from endlessly looping 
I got this which ate all my memory:

queuedaemon.php:10733] HTTPClient: HTTP HEAD https://drive.google.com/file/d/*masked*/view?usp=sharing - 200 OK
queuedaemon.php:10733] Checking for remote URL metadata for https://drive.google.com/file/d/*masked*/view?usp=sharing
queuedaemon.php:10733] HTTPClient: HTTP GET https://drive.google.com/file/d/*masked*/view?usp=sharing - 200 OK
queuedaemon.php:10733] Trying to discover an oEmbed endpoint using link headers.
queuedaemon.php:10733] Could not find an oEmbed endpoint using link headers, trying OpenGraph from HTML.
queuedaemon.php:10733] HTTPClient: HTTP HEAD https://drive.google.com/file/d/*masked*/view?usp=sharing&usp=embed_facebook - 200 OK
queuedaemon.php:10733] Checking for remote URL metadata for https://drive.google.com/file/d/*masked*/view?usp=sharing&usp=embed_facebook
queuedaemon.php:10733] HTTPClient: HTTP GET https://drive.google.com/file/d/*masked*/view?usp=sharing&usp=embed_facebook - 200 OK
queuedaemon.php:10733] Trying to discover an oEmbed endpoint using link headers.
queuedaemon.php:10733] Could not find an oEmbed endpoint using link headers, trying OpenGraph from HTML.
queuedaemon.php:10733] HTTPClient: HTTP HEAD https://drive.google.com/file/d/*masked*/view?usp=sharing&usp=embed_facebook&usp=embed_facebook - 200 OK
queuedaemon.php:10733] Checking for remote URL metadata for https://drive.google.com/file/d/*masked*/view?usp=sharing&usp=embed_facebook&usp=embed_facebook
queuedaemon.php:10733] HTTPClient: HTTP GET https://drive.google.com/file/d/*masked*/view?usp=sharing&usp=embed_facebook&usp=embed_facebook - 200 OK
queuedaemon.php:10733] Trying to discover an oEmbed endpoint using link headers.
queuedaemon.php:10733] Could not find an oEmbed endpoint using link headers, trying OpenGraph from HTML.
...ad nauseam.
 2017-05-01 10:27:21 +02:00
Mikael Nordfeldth
cb5bcf4937 bad log constant 2016-08-28 00:16:31 +02:00
Mikael Nordfeldth
1d791f81fa Attachment styling 2016-08-21 09:31:26 +02:00
Mikael Nordfeldth
563b3b1328 Using File->setTitle in oEmbed 2016-07-23 21:01:28 +02:00
Mikael Nordfeldth
1b3d583418 file_quota for OembedPlugin too 
Don't download huge files that might kill memory limits.
 2016-07-21 03:19:05 +02:00
Mikael Nordfeldth
809e2f6d07 Use File->getID() 2016-07-21 01:38:48 +02:00
Mikael Nordfeldth
ce65fe96ad Oembed bugs with thumbnail generation. 2016-04-18 15:33:20 +02:00
Mikael Nordfeldth
dcffe5d992 Forgotten File::getByUrl conversations (performance++) 2016-03-29 12:13:53 +02:00
Mikael Nordfeldth
97f7e6632d Embed attachments marked up as microformats2 (I think rather properly) 2016-03-27 16:21:43 +02:00
Mikael Nordfeldth
53c1750f0d If the attachment is a photo, don't replace representation in oEmbed 2016-03-22 14:02:36 +01:00
Mikael Nordfeldth
241b965715 oEmbed CSS file 2016-03-21 03:12:24 +01:00
Mikael Nordfeldth
365f3d2aa5 Full-content oEmbed html doesn't take up all space (and renders properly) 2016-03-17 12:58:40 +01:00
Mikael Nordfeldth
102f7ab059 oEmbed neatifying (inspired by Qvitter) 2016-03-17 00:31:45 +01:00
Mikael Nordfeldth
5ca2a28246 Make oEmbed handle our http/https setting better. 2016-03-10 14:20:21 +01:00
Mikael Nordfeldth
265fa12917 Relatively experimental change to store thumbnails in 'file/thumb/' (by default) 2016-03-07 22:33:34 +01:00
mmn
dc51354316 Merge branch 'readme-plugins' into 'nightly' 
Readme plugins

* Adds several plugin READMEs
  They are pretty basic, but it's a start.

* Changes status.net/wiki URLs to git.gnu.io
  The status.net wiki is dead.

See merge request !103
 2016-02-26 12:32:42 +00:00
Mikael Nordfeldth
4d17d95335 Try to get mime data before hashing (cpu intensive) 2016-02-25 22:31:45 +01:00
Mikael Nordfeldth
4239c952d2 $metadata->thumbnail_url is not guaranteed to be set 
We should probably have a separate class for this, so we can more
easily combine different technologies similar to oEmbed/OpenGraph.
 2016-02-25 19:46:17 +01:00
Mikael Nordfeldth
b9d35659c8 Stricter exception check 2016-02-10 04:43:30 +01:00
Chimo
a614205663 Add plugin READMEs 2016-02-08 17:48:37 +00:00
Mikael Nordfeldth
fb7f572eed Purify oembed html (again) 
For a commit or two we didn't do this, because htmLawed failed to filter
out CDATA javascript properly, but now we use HTML Purifier which works.
 2016-01-28 19:02:16 +01:00
Mikael Nordfeldth
7e6783bb8f Replace htmLawed with HTMLPurifier 2016-01-28 19:01:13 +01:00
Mikael Nordfeldth
daea5647b6 Merge branch 'nightly' of git.gnu.io:gnu/gnu-social into mmn_fixes 2016-01-28 17:27:08 +01:00
mmn
9b3cbb373e Merge branch 'oembed_fb_wp_branch' into 'nightly' 
Oembed: Fix UTF-8 bug and better wp&fb data (updated!)



See merge request !97
 2016-01-28 16:26:33 +00:00
Mikael Nordfeldth
efe23ed404 updateWithKeys now understands multi-column keys 
and automatically identifies _which_ columns are the right ones,
so for example 'uri' primary keys don't need to be explicitly set
 2016-01-28 16:42:59 +01:00
hannes
05439831e7 add comment that DOMDocument('1.0', 'UTF-8') does not work 2016-01-28 15:32:11 +00:00
hannes
06e325d61b fixes two issues when the oembed thumbnail is blank 2016-01-28 15:19:29 +00:00
hannes
aa76e5863f don't mess upp charsets in oembed/og! check for utf-8 in http header and meta tags, and add prolog when loading html with DOMDocument() 2016-01-26 13:37:52 +00:00
hannes
b8d1e1f4a6 silence errors on these xpath queries 2016-01-26 11:28:24 +00:00
hannes
884aeb4d2e common_purify() doesn't remove wordpress' and facebook's javascript properly, maybe better to keep the data intact, and do strip_tags or something similar when using the data 2016-01-26 01:10:15 +00:00
hannes
473f893d04 detab 2016-01-26 01:07:44 +00:00
hannes
76c8139054 not pretty, but gives us better oembed data for wordpress and facebook 2016-01-26 01:05:53 +00:00
hannes
d0e2f8745d add a thumbnail to oembed response 2016-01-21 18:48:30 +00:00
mmn
44c10bb2aa Merge branch 'oembed_branch' into 'nightly' 
purify oembed html and don't allow cdata

hopefully we never need stuff in cdata

reason for this is that this link serves javascript in its oembed data: https://www.maketecheasier.com/switch-windows-10-to-linux/

see:
https://www.maketecheasier.com/wp-json/oembed/1.0/embed?url=https%3A%2F%2Fwww.maketecheasier.com%2Fswitch-windows-10-to-linux%2F

i don't feel we want that in our database.  

See merge request !79
 2016-01-15 13:11:35 +00:00
Mikael Nordfeldth
0caf0612d0 Make Twitter Media upload API v1.1 reach us 
Now we just have to accept the 'media' or 'media_data' (base64 encoded)
POST arguments instead of $_FILES uploads.
 2016-01-14 18:29:21 +01:00
Mikael Nordfeldth
53339ff463 Fake oEmbed version in OpenGraph return object 2016-01-14 02:09:12 +01:00
Mikael Nordfeldth
24d9d76644 OpenGraph image/thumbnail width and height 2016-01-13 22:07:39 +01:00
hannes
ee305891c4 purify oembed html 2016-01-13 16:03:38 +00:00
Mikael Nordfeldth
3720e37f06 property attribute could be null in meta tags of course 2016-01-13 14:24:00 +01:00
Mikael Nordfeldth
3658774429 Super-basic OpenGraph image preview support, "works for me" 2016-01-12 15:29:03 +01:00
Mikael Nordfeldth
d4be5349b3 think I have managed to show oEmbed images better now 2016-01-07 17:35:37 +01:00
Mikael Nordfeldth
b596391fcd Avoid having to check for notices without rendered copies in upgrade.php 
Always call the Notice->getRendered() function to get a rendered copy.
We could perhaps put some sanitation there too in the future
 2016-01-06 15:32:27 +01:00
Mikael Nordfeldth
6772d991ae Only provide Notice oEmbed data for local notices 2015-12-31 01:55:18 +01:00
Mikael Nordfeldth
feb6b636f4 File_oembed varchar to text changes 
No need to have text length limitations in the database for fields which
very well may be longer than what was previously set.
 2015-12-27 12:11:29 +01:00
Mikael Nordfeldth
334a0d56e7 Oembed slimmed to only do discovery (soon we get og: discovery too) 2015-11-30 02:06:04 +01:00
Mikael Nordfeldth
b7edac2610 HTTPClient get $params array and oEmbedHelper uses it 2015-11-30 01:28:18 +01:00
Stephen Paul Weber
a9b1b60a97 Refactor on File::processNew 
The code was so involved there was even a comment asking for a refactor.

Now, File_redirection::where always returns a nice File_redirection
object instead of an array or string or nothing.  The object is
either one which already existed or else a new, unsaved object.

Instead of duplicating "does it exist" checks everywhere, do it in
File_redirection::where.  You either get what exists or something to save.

An unsaved File_redirection may be paired with an unsaved File.
You will want to save the File first (using ->saveFile()) and put the
id in File_redirection#file_id before saving.
 2015-11-02 05:15:08 +00:00
digital dreamer
d7fd6bac72 Snapshot of the Transifex translation project - October 2015 2015-10-04 18:23:01 +02:00