Commit Graph

202 Commits

Author SHA1 Message Date
Diogo Peralta Cordeiro
5077b65d71 [Media] Fix several issues
[StoreRemoteMedia] Upgrade plugin to use the new Media system

API Changes:
- Added getters to File to better formalize the ideas of the commit "[Media] Fix issues with database file storage"

UI Changes:
- Now presented thumbnails are actual thumbnails (bug fix)
- Attachment actions have a slightly more extended behaviour

Many other minor bug fixes...
2021-07-16 19:44:42 +01:00
Diogo Peralta Cordeiro
8bbbb890e3 [Media] Fix issues with database file storage
Fixed file quota as well.

There can be more than one file for the same filehash IF the url are different.

Possible states:
  - A file with no url and with filename is a local file.
  - A file with an url but no filename is a remote file that wasn't fetched,
    not even the thumbnail.
  - A file with an url and filename is a fetched remote file (maybe just a
    thumbnail of it).
  - A file with no filename nor url is a redirect.

Routes:
  Given these states, updated routes so that an attachment can only be
  retrieved by id and a file by filehash.

Major API changes:
  File::getByHash now returns a yield of files

Major UI changes:
  - Now remote non stored files are presented.
  - /view became preferred
  - Redirects to remote originals are preferred.

Many other minor bug fixes...
2021-07-16 19:44:42 +01:00
Alexei Sorokin
e62e429302 [File] Fix file deletion violating foreign keys 2021-07-16 19:44:40 +01:00
Alexei Sorokin
b1b1d2af93 [DATABASE] Update "modified" in Managed_DataObject instead of a DBMS trigger
Instead of relying on the MariaDB's ON UPDATE CURRENT_TIMESTAMP trigger update
"modified" attributes in Managed_DataObject. Every raw query that needs
adjusting is adjusted, as they won't update "modified" automatically anymore.

The main goal behind this change is to fix "modified" updates on PostgreSQL.
2021-07-16 19:44:37 +01:00
Diogo Cordeiro
9f4c4edb02 [ActivityPub][Ostatus] Fetch avatars in accordance to the new media system 2021-07-16 19:44:37 +01:00
Alexei Sorokin
26115482ef [SCHEMA] Improve timestamp storage
Avoid the use of deprecated MariaDB "zero dates" globally. If they're present
as attribute defaults somewhere, they will be replaced with NULL implicitly.
The existing "zero dates" in MariaDB storage will be left intact and this
should not present any issues.

The "timestamp" type in table definitions now corresponds to DATETIME in
MariaDB with "DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP", which
should be close enough to the original behaviour for compatibility purposes.
It is now the recommended type for "modified" attributes, because of the
update trigger on MariaDB. But there is no such trigger implemented on
PostgreSQL as of this moment.
2021-07-16 19:44:37 +01:00
Diogo Cordeiro
f93d8694c3 [AVATAR] Ensure this Action stays secure 2021-07-16 19:44:37 +01:00
Alexei Sorokin
ed4cc01b2f [DATABASE] Replace NOW() with CURRENT_TIMESTAMP 2021-07-16 19:44:36 +01:00
Diogo Cordeiro
3dcedfa8bd [MEDIA] Simplify Attachment actions 2021-07-16 19:44:35 +01:00
Diogo Cordeiro
4a78510e37 [FILE] Fix Return value of File::getFileOrThumbnailSize() must be of the type int, null returned 2021-07-16 19:44:35 +01:00
Alexei Sorokin
16b5ddd230 [DATABASE] Re-introduce PostgreSQL support 2021-07-16 19:44:35 +01:00
Alexei Sorokin
d26aac77b3 [DATABASE] Always quote identifiers
The code used to operate under the assumption that MariaDB doesn't support
quoting identifiers. Not only is that not exactly true, but MariaDB has
reserved keywords that cannot be used as table or column names unquoted.
2021-07-16 19:44:35 +01:00
Diogo Cordeiro
0d7c0069f2 [MODULES] Allow to upload third party plugins
Fixed some bugs
2021-07-16 19:44:34 +01:00
Miguel Dantas
6d552f15b6 [Embed] Resize thumbnails fetched by Embed to avoid keeping original images when we need only 128x128. Size configurable in config.php 2019-08-03 17:49:36 +01:00
Diogo Cordeiro
f67a93eddc [CORE] Bump Database requirement to MariaDB 10.3+ 2019-08-03 17:47:23 +01:00
Miguel Dantas
7070a14480 [CORE][StoreRemoteMedia] Fixed bug where sometimes images were written outside the site root 2019-08-03 17:31:43 +01:00
Miguel Dantas
3e5ce46e98 [CORE] Fixed bug where the http connection was using the wrong size for thumbnails, and returning the wrong one 2019-08-03 17:31:42 +01:00
Miguel Dantas
f746866b65 [StoreRemoteMedia] StoreRemoteMedia now uses the new filename format, which allows it to display correctly in the UI. Formatting fixes 2019-08-03 17:31:42 +01:00
Miguel Dantas
8f31a1a820 [MEDIA][OEMBED] Fixed regression in OEmbed, because it relied on accessing the files directly, which previous commits broke. The File table really should have a bool... 2019-08-03 17:31:42 +01:00
Miguel Dantas
da82048d77 [MEDIA] Fix trying to display file wich is not available locally 2019-08-03 17:31:41 +01:00
Miguel Dantas
aa5c6bbf08 [CORE][UI] Made attachment actions and its subactions be able to identify attachments by id and by filehash. Changed the url stored in the DB to be attachment//view 2019-08-03 17:31:41 +01:00
Miguel Dantas
c7475d78b4 [CORE][UI][ROUTER] Added view action, which inlines images and videos but downloads everything else. Fixed File url to get an URL fromthe view action, so when a making a remote notice, the correct URL is used, not accessing directly to the file 2019-08-03 17:31:40 +01:00
Miguel Dantas
3c9a07677e [CORE] Attachments and thumbnails aren't accessed directly by the file under the file storage folder, but indirectly from PHP, so that access to the file folder can be blocked in the server config 2019-08-03 17:31:40 +01:00
Miguel Dantas
ccebe536b3 [MEDIA] Removed blacklisted extensions, "trusts" upload extension (doesn't affect anything) and updated sysadmin documentation
Fixes bug which broke the UI. Oops
2019-08-03 17:31:39 +01:00
Miguel Dantas
5eb4a7d711 [MEDIA] File downloader now in PHP, added proper name in the UI and changed the format for new attachment file names
The file downloader was changed from a simple redirect to the file to one
implemented in PHP, which should make it safer, by making it possible disallow
direct access to the file, to prevent executing of atttachments

The filename has a new format:
  bin2hex("{$original_name}")."-{$filehash}"
This format should be respected. Notice the dash, which is important to distinguish it from the previous
format, which was "{$hash}.{$ext}"

This change was made to both make the experience more user friendly, by
providing a readable name for files, as opposed to it's hash. This name is taken
from the upload filename, but, clearly, as this wasn't done before, it's
impossible to have a proper name for older files, so those are displayed as
"untitled.{$ext}".

This new name is displayed in the UI, instead of the previous name.
2019-08-03 17:31:36 +01:00
Miguel Dantas
20c73f0596 [FORMATING] Ran php-cs-fixer on lib/mediafile.php, lib/imagefile.php and classes/File.php 2019-06-10 00:38:16 +01:00
Miguel Dantas
b224d93098 [MEDIA] ImageFile now extends MediaFile and validates images more aggressively.
Default supported files need to use consistent names. Bumped version to 1.20.0

ImageFile has been changed to extend MediaFile and rely on it to partially
validate files. This validation has been extended to not rely solely on
Fileinfo, as it is disabled on some places. Now it'll try to use the shell
command `file`, if Fileinfo isn't available.

ImageFile now converts every new upload to PNG, except JPEG and GIF, which
are kept, but still resized (to the same size), to remove possible scripts
embedded therein.

MediaFile::fromUpload will return an ImageFile if the uploaded file is an image
or a MediaFile otherwise.

MediaFile can be constructed with an id with value -1 to denote a temporary
object, which is not added to the DB. This is useful to create a temporary
object for representing images, so it can be used to rescale them.

The supported attachment array needs to be populated with the result of calling
`image_type_to_extension` for the appropriate image type, in the case of images.
This is important so all parts of the code see the same extension for each image
type (jpg vs jpeg).

Added documentation to classes/File.php and to lib/MediaFile and lib/ImageFile
2019-06-10 00:35:53 +01:00
Mikael Nordfeldth
586aaa596e Quote URL on dupe search and better logging on upgrade 2017-07-27 19:39:55 +02:00
Mikael Nordfeldth
979c525124 I like to throw exceptions instead of using if statements. 2017-05-02 09:07:39 +02:00
Mikael Nordfeldth
d84bf83419 Created function File->setTitle(str) 2016-07-23 21:00:57 +02:00
Mikael Nordfeldth
f02d32b718 Reworked File->getUrl to throw exception
In case you require a local URL and one can't be generated, throw
FileNotStoredLocallyException(File $file)
2016-07-07 00:44:50 +02:00
Mikael Nordfeldth
b4a0bff740 Some mimetype madness! 2016-07-06 08:59:16 +02:00
Mikael Nordfeldth
7978cd6d59 s/EmptyIdException/EmptyPkeyValueException/ 2016-06-25 11:50:59 +02:00
Mikael Nordfeldth
1dfac3ad63 Allow getting filesize by function 2016-06-24 15:53:23 +02:00
Mikael Nordfeldth
6d6db77f06 Documentation update in File class file 2016-05-04 11:44:14 +02:00
Mikael Nordfeldth
87dd0fbdb6 UseFileAsThumbnailException uses direct File object now 2016-05-04 11:34:50 +02:00
Mikael Nordfeldth
7aa9a69c2f Link to attachment page instead of big-ass image 2016-05-01 11:35:51 +02:00
Mikael Nordfeldth
15c16ac54e Make sure File_thumbnail->getPath() doesn't throw NoResultException 2016-03-09 23:49:01 +01:00
Mikael Nordfeldth
a24c6fdb39 If our host matched in File lookup, it could throw exceptions on Router->map 2016-03-09 14:52:15 +01:00
Mikael Nordfeldth
3b1181dae6 Display locally stored remote GIFs as thumbnails
Many newspapers seem to use animated GIFs as catchy header images, which
we would fail to show from oEmbed/OpenGraph fetching since they would
want us to "use File as Thumbnail", but the only place the image filename
was stored was in File_thumbnail, for the thumbnail of that file_id which
had a URL set.
2016-03-08 01:36:24 +01:00
Mikael Nordfeldth
265fa12917 Relatively experimental change to store thumbnails in 'file/thumb/' (by default) 2016-03-07 22:33:34 +01:00
Mikael Nordfeldth
a3b265a477 Portability for filepath in File 2016-03-07 20:13:07 +01:00
Mikael Nordfeldth
63c087a255 Consistent behaviour for ScopingNoticeStream $scoped
We don't guess the current profile anymore if the value of the profile === -1

Also sets $this->scoped for all ScopingNoticeStream inheritors, which just
like in an Action can be null if we're not scoped in any way (logged in).
2016-03-01 14:51:47 +01:00
Mikael Nordfeldth
1e6520fddd Woops, forgot to skip the / in path 2016-02-26 14:13:46 +01:00
Mikael Nordfeldth
6a4470912f Fiddling with merge request #98 to use internal routing functions 2016-02-26 14:10:32 +01:00
Mikael Nordfeldth
8356c2495c Use mb_* and strict === comparison 2016-02-26 13:52:25 +01:00
mmn
722ff4d9c0 Merge branch 'foolproof_file_redirection_branch' into 'nightly'
Foolproof file redirection

This solves an issue when our internal /attachment/{file_id} links are shortened with an remote shorteners (which caused the /attachment/{file_id} links to be saved to the File table and a thumbnail of a thumbnail being generated)

See merge request !98
2016-02-26 12:49:10 +00:00
Mikael Nordfeldth
aeb2e282db Commented on the mime extension matching regexp 2016-02-25 22:32:54 +01:00
Mikael Nordfeldth
bac37d1714 syntax error 2016-02-25 22:17:44 +01:00
Mikael Nordfeldth
e6e1705852 Make uploads work properly if we accept _all_ attachment types
Also introduced $config['attachments']['extblacklist'] that can disable
certain file extensions (or rewrite them, for example php => phps)
2016-02-25 22:15:54 +01:00