Commit graph

1365 commits

Author SHA1 Message Date
Eugen Rochko
9ba40a6bfd
Remove HEAD request from fetching link previews (#12028)
It is not really necessary and we need to reduce requests
2019-10-01 04:54:10 +02:00
Eugen Rochko
5c42f47617
Fix records not being indexed sometimes (#12024)
It's possible that after commit callbacks were not firing when
exceptions occurred in the process. Also, the default Sidekiq
strategy does not push indexing jobs immediately, which is not
necessary and could be part of the issue too.
2019-10-01 01:19:11 +02:00
Eugen Rochko
5f69eb89e2
Add a nodeinfo endpoint (#12002)
* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* Add some additional node info stuff

* Add nodeinfo endpoint

* dont commit stuff from my local dev

* consistant naming since we implimented 2.1 schema

* expanding this to include federation info

* codeclimate feedback

* CC feedback

* using activeserializers seems like a good idea...

* get rid of draft 2.1 version

* Reimplement 2.1, also fix metaData -> metadata

* Fix metaData -> metadata here too

* Fix nodeinfo 2.1 tests

* Implement cache for monthly user aggregate

* Useless

* Remove ostatus from the list of supported protocols

* Fix nodeinfo's open_registration reading obsolete setting variable

* Only serialize domain blocks with user-facing limitations

* Do not needlessly list noop severity in nodeinfo

* Only serialize domain blocks info in nodeinfo when they are set to be displayed to everyone

* Enable caching for nodeinfo endpoints

* Fix rendering nodeinfo

* CodeClimate fixes

* Please CodeClimate

* Change InstancePresenter#active_user_count_months for clarity

* Refactor NodeInfoSerializer#metadata

* Remove nodeinfo 2.1 support as the schema doesn't exist

* Clean-up
2019-09-29 21:31:51 +02:00
Eugen Rochko
ab33c4df94
Add exclude_unreviewed param to GET /api/v2/search REST API (#11977)
Make it so normal search returns even unreviewed matches, but
autosuggestions do not.

Fix #11960
2019-09-28 01:02:21 +02:00
ThibG
18b451c0e6 Change silences to always require approval on follow (#11975)
* Change silenced accounts to require approval on follow

* Also require approval for follows by people explicitly muted by target accounts

* Do not auto-accept silenced or muted accounts when switching from locked to unlocked

* Add `follow_requests_count` to verify_credentials

* Show “Follow requests” menu item if needed even if account is locked

* Add tests

* Correctly reflect that follow requests weren't auto-accepted when local account is silenced

* Accept follow requests from user-muted accounts to avoid leaking mutes
2019-09-27 21:13:51 +02:00
Eugen Rochko
3ed94dcc1a
Add account migration UI (#11846)
Fix #10736

- Change data export to be available for non-functional accounts
- Change non-functional accounts to include redirecting accounts
2019-09-19 20:58:19 +02:00
Eugen Rochko
e1066cd431
Add password challenge to 2FA settings, e-mail notifications (#11878)
Fix #3961
2019-09-18 16:37:27 +02:00
Eugen Rochko
4f6af87906
Change spam check to apply to local accounts and add a threshold (#11806)
Instead of detecting spam on first duplicate message, add a
threshold of 5 such messages to reduce false positives
2019-09-18 12:53:13 +02:00
Eugen Rochko
a4b60e9ba4
Fix TOTP codes not being filtered from logs during enabling/disabling (#11877)
Not a serious issue because they are meaningless past single use
2019-09-18 02:48:40 +02:00
Eugen Rochko
c707ef49d9
Fix 2FA challenge and password challenge for non-database users (#11831)
* Fix 2FA challenge not appearing for non-database users

Fix #11685

* Fix account deletion not working when using external login

Fix #11691
2019-09-15 21:08:39 +02:00
Eugen Rochko
18331fefa2
Remove deprecated GET /api/v1/search API (#11823)
Use `GET /api/v2/search` instead
2019-09-13 16:11:13 +02:00
Eugen Rochko
0762258aec
Fix hashtags being split by ZWNJ character (#11821)
Fix #11761
2019-09-13 16:01:26 +02:00
Eugen Rochko
c5d37f18cb
Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Tao Bror Bojlén
4fe127664b add admin setting for default search engine indexing (fix #11750) (#11804) 2019-09-11 08:44:58 +02:00
ThibG
4faaa5b25e Add updated relationship to follow request API responses (#11800)
Fixes #11747
2019-09-10 20:56:42 +02:00
Eugen Rochko
1110ea1a91
Add batch actions and categories to admin UI for custom emojis (#11793) 2019-09-09 22:44:17 +02:00
Eugen Rochko
e445a8af64
Add timeline read markers API (#11762)
Fix #4093
2019-09-06 13:55:51 +02:00
ThibG
692c5b439a Fix ActivityPub context not being dynamically computed (#11746)
* Fix contexts not being dynamically included

Fixes #11649

* Refactor Note context in serializer

* Refactor Actor serializer
2019-09-03 22:52:32 +02:00
Eugen Rochko
70ddef2654
Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
Eugen Rochko
b54b725d6b
Fix uncaught domain normalization error in remote follow (#11703) 2019-08-30 02:19:17 +02:00
Eugen Rochko
22ce4778eb
Fix uncaught parameter missing exceptions and missing error templates (#11702) 2019-08-30 01:34:47 +02:00
Eugen Rochko
73ca0bb925
Add option to include reported statuses in warning e-mail (#11639) 2019-08-23 22:37:23 +02:00
Eugen Rochko
97192d9a77
Fix remote and staff-removed statuses leaving media behind for a day (#11638)
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
2019-08-22 04:17:12 +02:00
Eugen Rochko
cc0a55cf9a
Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
2019-08-18 03:45:51 +02:00
Eugen Rochko
e5cee8062f
Fix blurhash and autoplay not working on public pages (#11585) 2019-08-16 19:15:05 +02:00
Eugen Rochko
8fdff2748f
Add more accurate account search (#11537)
* Add more accurate account search

When ElasticSearch is available, a more accurate search is implemented:

- Using edge n-gram index for acct and display name
- Using asciifolding and cjk width normalization on display names
- Using Gaussian decay on account activity for additional scoring (recency)
- Using followers/friends ratio for additional scoring (spamminess)
- Using followers number for additional scoring (size)

The exact match precedence only takes effect when the input conforms
to the username format and the username part of it is complete, i.e.
when the user started typing the domain part.

* Support single-letter usernames

* Fix tests

* Fix not picking up account updates

* Add weights and normalization for scores, skip zero terms queries

* Use local counts for accounts index, adjust search parameters

* Fix mistakes

* Using updated_at of accounts is inadequate for remote accounts
2019-08-16 01:24:03 +02:00
ThibG
bced70469a Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Yusuke Nakamura
82d2069c75 Bump faker from 1.9.6 to 2.1.0 and update faker api (#11489)
* Bump faker from 1.9.6 to 2.1.0

Bumps [faker](https://github.com/stympy/faker) from 1.9.6 to 2.1.0.
- [Release notes](https://github.com/stympy/faker/releases)
- [Changelog](https://github.com/stympy/faker/blob/master/CHANGELOG.md)
- [Commits](https://github.com/stympy/faker/compare/1.9.6...v2.1.0)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Use faker api v2

https://github.com/stympy/faker/releases/tag/2.0
2019-08-06 15:33:03 +02:00
Eugen Rochko
115dab78f1
Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
2019-08-05 19:54:29 +02:00
Eugen Rochko
8b9d0a0533
Remove XML version of Webfinger and remove links to Atom feeds (#11460)
Fix #11453
2019-08-01 19:14:02 +02:00
Eugen Rochko
92de439c04
Change hashtag search to only return results that have trended in the past (#11448)
* Change hashtag search to only return results that have trended in the past

A way to eliminate typos and other one-off "junk" results

* Fix excluding exact matches that don't have a score

* Fix tests
2019-07-30 20:29:50 +02:00
ThibG
ff789a751a Fix boosting & unboosting preventing a boost from appearing in the TL (#11405)
* Fix boosting & unboosting preventing a boost from appearing in the TL

* Add tests

* Avoids side effects when aggregate_reblogs isn't true
2019-07-30 13:18:23 +02:00
Eugen Rochko
24552b5160
Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Eugen Rochko
e136112ab7
Fix tag normalization and migration not removing duplicate tags (#11441)
Fix #11428
2019-07-29 20:40:21 +02:00
Eugen Rochko
b9b0313c78
Revert "Remove conversation URI (#11423)" (#11424)
This reverts commit 75f7f9930e.
2019-07-28 17:47:37 +02:00
Eugen Rochko
75f7f9930e
Remove conversation URI (#11423)
It is not part of ActivityPub and will free up a lot of space
2019-07-28 17:30:12 +02:00
ysksn
d6ada2eb30 Implement pending tests (#11415) 2019-07-27 10:24:26 +02:00
Eugen Rochko
b9fbcbfe4e
Add search syntax for operators and phrases (#11411) 2019-07-27 04:42:08 +02:00
ThibG
92569ffde8 Fix invites not being disabled upon account suspension (#11412)
* Disable invite links from disabled/suspended users

* Add has_many invites relationship to users

* Destroy unused invites when suspending an account
2019-07-26 18:55:33 +02:00
Rey Tucker
94f5c714f1 Don't delete periods when validating username uniqueness (#11392) (#11400)
* Check to make sure usernames with '.' cannot be created

* Add test for instance actor account name conflicts

This makes sure that migration 20190715164535_add_instance_actor
won't fail if there's already an account that is named the same
as the domain (minus the .)

* Put the test into the correct context...

* Add another test to split this into two validations

* Don't delete periods when validating username uniqueness (#11392)

The 20190715164535_add_instance_actor migration fails if there's
already a username similar to the domain name, e.g. if you are
'vulpine.club' and have a user named 'vulpineclub', validation
fails.

Upon further review, usernames with periods are dropped by the
regular expression in the Account class, so we don't need to
worry about it here.

Fixes #11392
2019-07-24 14:19:17 +02:00
Eugen Rochko
964ae8eee5
Change unconfirmed user login behaviour (#11375)
Allow access to account settings, 2FA, authorized applications, and
account deletions to unconfirmed and pending users, as well as
users who had their accounts disabled. Suspended users cannot update
their e-mail or password or delete their account.

Display account status on account settings page, for example, when
an account is frozen, limited, unconfirmed or pending review.

After sign up, login users straight away and show a simple page that
tells them the status of their account with links to account settings
and logout, to reduce onboarding friction and allow users to correct
wrongly typed e-mail addresses.

Move the final sign-up step of SSO integrations to be the same
as above to reduce code duplication.
2019-07-22 10:48:50 +02:00
ThibG
7de8c51873 Play animated custom emoji on hover (#11348)
* Play animated custom emoji on hover in status

* Play animated custom emoji on hover in display names

* Play animated custom emoji on hover in bios/bio fields

* Add support for animation on hover on public pages emojis too

* Fix tests

* Code style cleanup
2019-07-21 18:10:40 +02:00
Eugen Rochko
bd1545de5e
Change locale detection to run once per session (#8657)
Fix #6462
2019-07-21 18:08:02 +02:00
Eugen Rochko
bd87e66679
Remove WebSub subscriptions (#11303) 2019-07-21 04:08:00 +02:00
ThibG
c37c1da41e Disallow numeric-only hashtags (#11363)
* Add spec covering numeric-only hashtags

* Fix hashtag regex
2019-07-19 23:22:35 +02:00
ThibG
fda437a020 Fix sanitizing lists contents (#11354)
* Add test

* Fix code for sanitizing nested lists stripping all tags
2019-07-19 01:44:58 +02:00
ThibG
730c4053d6 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
2019-07-19 01:44:42 +02:00
Eugen Rochko
84e988479e
Fix only one middle dot being recognized in hashtags (#11345)
Fix #10934
2019-07-18 03:02:56 +02:00
Eugen Rochko
5bfe1e1f05
Change language detection to include hashtags as words (#11341) 2019-07-18 03:02:15 +02:00
Eugen Rochko
6ff67be0f6
Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
2019-07-13 16:45:50 +02:00
Eugen Rochko
5bf67ca913
Add ActivityPub secure mode (#11269)
* Add HTTP signature requirement for served ActivityPub resources

* Change `SECURE_MODE` to `AUTHORIZED_FETCH`

* Add 'Signature' to 'Vary' header and improve code style

* Improve code style by adding `public_fetch_mode?` method
2019-07-11 20:11:09 +02:00
Eugen Rochko
4e8dcc5dbb
Add HTTP signatures to all outgoing ActivityPub GET requests (#11284) 2019-07-11 14:49:55 +02:00
Eugen Rochko
5d3feed191
Refactor fetching of remote resources (#11251) 2019-07-10 18:59:28 +02:00
Eugen Rochko
4e92183227
Refactor domain block checks (#11268) 2019-07-09 03:27:35 +02:00
Eugen Rochko
ef15246397
Remove unused remote unfollow controller (#11250) 2019-07-08 12:04:06 +02:00
Eugen Rochko
63c7fe8e48
Refactor controllers for statuses, accounts, and more (#11249) 2019-07-08 12:03:45 +02:00
Eugen Rochko
b851456139
Remove Atom feeds and old URLs in the form of GET /:username/updates/:id (#11247) 2019-07-07 16:16:51 +02:00
Eugen Rochko
23aeef52cc
Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Eugen Rochko
0c1b1069c9
Remove deprecated REST API GET /api/v1/statuses/:id/card (#11213) 2019-07-05 02:15:24 +02:00
Eugen Rochko
3fd6ab99e6
Remove deprecated REST API GET /api/v1/timelines/direct (#11212) 2019-07-05 02:14:56 +02:00
Eugen Rochko
0d9ffe56fb
Add request pool to improve delivery performance (#10353)
* Add request pool to improve delivery performance

Fix #7909

* Ensure connection is closed when exception interrupts execution

* Remove Timeout#timeout from socket connection

* Fix infinite retrial loop on HTTP::ConnectionError

* Close sockets on failure, reduce idle time to 90 seconds

* Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server

* Use a shared pool size, 512 by default, to stay below open file limit

* Add some tests

* Add more tests

* Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds

* Use a shared pool that returns preferred connection but re-purposes other ones when needed

* Fix wrong connection being returned on subsequent calls within the same thread

* Reduce mutex calls on flushes from 2 to 1 and add test for reaping
2019-07-02 00:34:38 +02:00
Eugen Rochko
e64e6a03dd
Add categories for custom emojis (#11196)
Fix #7940
2019-06-28 15:54:10 +02:00
ThibG
9a90ec3b3b Fix account URI in UpdatePollSerializer (#11194)
* Fix account URI in UpdatePollSerializer

Fixes #11185

* Add specs
2019-06-27 19:41:55 +02:00
ThibG
915c619394 Add support for Audio activities (#11189)
Fixes #11127
2019-06-26 19:32:36 +02:00
ThibG
47ef4a6c7a Apply filters to poll options (#11174)
* Apply filters to poll options in WebUI

Fixes #11128

* Apply filters to poll options server-side

* Add poll options to searchable text
2019-06-25 14:45:14 +02:00
Eugen Rochko
707ddf7808
Change domain blocks to automatically support subdomains (#11138)
* Change domain blocks to automatically support subdomains

If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)

* Match subdomains of existing accounts when blocking/unblocking domains

* Improve code style
2019-06-22 00:13:10 +02:00
Eugen Rochko
7696f77245
Add moderation API (#9387)
Fix #8580
Fix #7143
2019-06-20 02:52:34 +02:00
Eugen Rochko
103a9f4466
Fix sanitizer making block level elements unreadable (#10836)
Fix #10834
2019-06-16 21:46:36 +02:00
Eugen Rochko
560ec24e58
Change /settings/preferences to redirect to appearance, add /settings/preferences/other (#10988) 2019-06-07 16:51:08 +02:00
Eugen Rochko
1db4117030
Change preferences page into appearance, notifications, and other (#10977) 2019-06-07 03:39:24 +02:00
ThibG
6c464cd424 Do not misattribute inlined boosts if attributedTo isn't present (#10967)
* Do not misattribute inlined boosts if `attributedTo` isn't present

Fixes #10950

* Fix tests
2019-06-04 23:24:31 +02:00
Eugen Rochko
48fee1a800
Fix poll API not requiring authentication on non-public polls (#10960)
* Fix poll API not requiring authentication on non-public polls

That API does not reveal the content of the status, i.e. the question
itself, nor who the author is, nor which status it belongs to, but it
does reveal the poll options and how many answers they got

Fix #10959

* Add test
2019-06-04 20:10:26 +02:00
ThibG
a353dc6030 Fix NotifyService test with regards to reblogs (#10928)
Fixes #10890
2019-06-02 18:08:26 +02:00
trwnh
e3b39ea4a4 Update remote bio test from 160 to 500 (#10799) 2019-05-21 13:29:06 +02:00
Paul Woolcock
0c933c1b8c Add account_id param to GET /api/v1/notifications (#10796)
* Add `from_account` to notifications API

this adds the ability to filter notifications by the account they
originated from

* passing a non-existent user should cause none to be returned

* Fix codeclimate warnings

* fix more codeclimate warnings

* make requested changes:

* use account id instead of user@domain
* name the param `account_id` instead of `from_account`

* Don't use `return` in a lambda
2019-05-21 13:28:49 +02:00
trwnh
a6caf919e2 Change bio limit from 160 to 500 (#10790)
* Change note_length validator from 160 to 500

* Change input maxlength from 160 to 500

* update bio test from 160 to 500

* Multiply a string 30 times instead of 10
2019-05-19 22:51:44 +02:00
ThibG
ae18386558 Fix “invited by” not showing up for invited accounts in admin interface (#10791) 2019-05-19 21:40:36 +02:00
ThibG
a1519a8ef5 Prevent from publicly boosting one's own private toots (#10775) 2019-05-18 00:28:51 +02:00
ThibG
14f6ce2885 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
ThibG
62f5235b6f Prevent silenced local users from notifying remote users not following them (#10575)
* Prevent silenced local users from notifying remote users not following them

This is an attempt to extend the local restrictions of silenced users to the
federation.

* Add tests

* Add tests for making sure private status don't get sent over OStatus
2019-05-09 22:05:43 +02:00
Eugen Rochko
7cb369d4c6
Change e-mail whitelist/blacklist to not be checked when invited (#10683)
* Change e-mail whitelist/blacklist to not be checked when invited

And only when creating an account, not when updating it later

Fix #10648

* Fix test
2019-05-03 23:44:44 +02:00
ThibG
011b032300 Provide a link to existing domain block when trying to block an already-blocked domain (#10663)
* When trying to block an already-blocked domain, provide a link to the block

* Fix styling for links in flash messages

* Allow blocks to be upgraded but not downgraded
2019-05-03 20:36:36 +02:00
ThibG
21a73c52a7 Check that an invite link is valid before bypassing approval mode (#10657)
* Check that an invite link is valid before bypassing approval mode

Fixes #10656

* Add tests

* Only consider valid invite links in registration controller

* fixup
2019-05-02 04:30:12 +02:00
Eugen Rochko
a9f130b8d8
Fix Keybase verification using wrong domain for remote accounts (#10547) 2019-04-10 20:28:43 +02:00
Alex Gessner
154106c0c3 compare usernames case-insensitively on new proof creation flow (#10544)
* compare usernames case-insensitively on new proof creation flow

* Fix code style issue
2019-04-10 18:05:11 +02:00
Eugen Rochko
46cb36fd2c
Add invite request to pending account notification e-mail (#10528)
Fix sorting of the pending accounts page
2019-04-10 00:36:01 +02:00
Hinaloe
48f466daf1 Allow set the voting period to just 5 minutes (#10525)
* Add spec of PollValidator for #10190

* Raise fraction less than 1 second

* format

* simplify time initialize
2019-04-09 17:02:12 +02:00
Eugen Rochko
8b69a66380 Add "why do you want to join" field to invite requests (#10524)
* Add "why do you want to join" field to invite requests

Fix #10512

* Remove unused translations

* Fix broken registrations when no invite request text is submitted
2019-04-09 23:06:30 +09:00
ThibG
cb71c95e22 Export and import show_reblogs together with following list (#10495)
* Refactor imports

* Export show_reblogs when exporting list of followed users

* Add support for importing show_reblogs with following collection

* Fix tests
2019-04-08 07:28:27 +02:00
Eugen Rochko
67b3b62b98
Improve blocked view of profiles (#10491)
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"

This reverts commit 120544067f.

* Revert "Hide blocking accounts from blocked users (#10442)"

This reverts commit 62bafa20a1.

* Improve blocked view of profiles

- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
2019-04-07 04:59:13 +02:00
ThibG
d4882aa64a Export and import hide_notifications alongside user mutes (#10335)
* Export hide_notifications along with user mutes

* Import hide_notifications along with muted users list

* Add headers for CSV exports
2019-04-03 18:17:43 +02:00
ThibG
62bafa20a1 Hide blocking accounts from blocked users (#10442)
* Revert "Add indication that you have been blocked in web UI (#10420)"

This reverts commit bd02ec6daa.

* Revert "Add `blocked_by` relationship to the REST API (#10373)"

This reverts commit 9745de883b.

* Hide blocking accounts from search results

* Filter blocking accouts from account followers

* Filter blocking accouts from account's following accounts

* Filter blocking accounts from “reblogged by” and “favourited by” lists

* Remove blocking account from URL search

* Return 410 on trying to fetch user data from a user who blocked us

* Return 410 in /api/v1/account/statuses for suspended or blocking accounts

* Fix status filtering when performing URL search

* Restore some React improvements

Restore some cleanup from bd02ec6daa

* Refactor by adding `without_blocking` scope
2019-04-01 20:06:13 +02:00
ThibG
2acd8940de Fix more keybase-related test failures (#10443) 2019-04-01 19:10:44 +02:00
slice
85973f4f37 Improvements to image upload validation and creation (#10431)
* Check if image value is nil? before creating an image

Check if uploaded images aren't nil before creating SiteUpload models
for them.

* Validate presence of file in SiteUpload

* Fix file presence validation

* Fabricate SiteUpload#file

* Add link to Creative Commons license
2019-04-01 07:30:46 +02:00
ThibG
abecaba317 Fix failing keybase-related test (#10428) 2019-03-30 18:15:23 +01:00
Eugen Rochko
1714ea5978
Add ActivityPub representation for identity proofs (#10414)
* Add ActivityPub representation for identity proofs

* Add tests
2019-03-30 02:12:06 +01:00
Alex Gessner
69141dca26 squashed identity proof updates (#10375) 2019-03-28 18:01:09 +01:00
Eugen Rochko
f1bc90ab50
Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
2019-03-28 04:44:59 +01:00
Eugen Rochko
11fe293e1b
Remove unused ActivityPub @context values depending on response (#10378)
Fix #8078
2019-03-27 15:55:23 +01:00
Eugen Rochko
555c4e11ba
Add validations to admin settings (#10348)
* Add validations to admin settings

- Validate correct HTML markup
- Validate presence of contact username & e-mail
- Validate that all usernames are valid
- Validate that enums have expected values

* Fix code style issue

* Fix tests
2019-03-23 14:07:04 +01:00
ThibG
66d9452092 Do not try fetching keys of unknown accounts on a Delete from them (#10326) 2019-03-20 17:20:16 +01:00
Eugen Rochko
9c4cbdbafb
Add Keybase integration (#10297)
* create account_identity_proofs table

* add endpoint for keybase to check local proofs

* add async task to update validity and liveness of proofs from keybase

* first pass keybase proof CRUD

* second pass keybase proof creation

* clean up proof list and add badges

* add avatar url to keybase api

* Always highlight the “Identity Proofs” navigation item when interacting with proofs.

* Update translations.

* Add profile URL.

* Reorder proofs.

* Add proofs to bio.

* Update settings/identity_proofs front-end.

* Use `link_to`.

* Only encode query params if they exist.

URLs without params had a trailing `?`.

* Only show live proofs.

* change valid to active in proof list and update liveness before displaying

* minor fixes

* add keybase config at well-known path

* extremely naive feature flagging off the identity proof UI

* fixes for rubocop

* make identity proofs page resilient to potential keybase issues

* normalize i18n

* tweaks for brakeman

* remove two unused translations

* cleanup and add more localizations

* make keybase_contacts an admin setting

* fix ExternalProofService my_domain

* use Addressable::URI in identity proofs

* use active model serializer for keybase proof config

* more cleanup of keybase proof config

* rename proof is_valid and is_live to proof_valid and proof_live

* cleanup

* assorted tweaks for more robust communication with keybase

* Clean up

* Small fixes

* Display verified identity identically to verified links

* Clean up unused CSS

* Add caching for Keybase avatar URLs

* Remove keybase_contacts setting
2019-03-18 21:00:55 +01:00
ThibG
a20354a20b Set and store report URIs (#10303)
Fixes #10271
2019-03-17 15:34:56 +01:00
ThibG
5e38ef87a7 Fix reblogs privacy (#10302)
* Fix reblogs privacy

* Fix Announce processing specs
2019-03-17 14:54:09 +01:00
Eugen Rochko
1c113fd72d
Add relationship manager UI (#10268) 2019-03-16 11:23:22 +01:00
ysksn
782b622f5f Add specs for action log helper (#9605)
* Add specs for ActionLogHelper

* Make some methods private

methods below never referenced from outside of their module:

- #linkable_log_target
- #log_target_from_history
2019-03-16 00:57:23 +09:00
Eugen Rochko
1b167707c2
Fix language detection of non-latin alphabets even at few characters (#10276) 2019-03-15 05:07:09 +01:00
Eugen Rochko
51e154f5e8
Admission-based registrations mode (#10250)
Fix #6856
Fix #6951
2019-03-14 05:28:30 +01:00
Eugen Rochko
65fffeac3f
Redesign landing page (#10232) 2019-03-12 17:34:00 +01:00
Aurélien Reeves
85537b0069 Squish username before validation (#10239)
* Squish username before validation (#10101)

Fix #10101

* Move before_validation hook to a private method

Also add Unicode wite-spaces to the spec to support the use of squish
over strip.
2019-03-11 20:48:24 +01:00
ThibG
c11dff5049 Reject existing Follows when suspending a remote account (#10230)
* Reject existing Follows when suspending a remote account

Partial fix to #10229

* Add tests
2019-03-10 16:18:58 +01:00
ThibG
3aaac4f134 Do not allow adding votes to expired polls (#10214)
* Do not allow adding votes to expired polls

* Only validate expires_at on create
2019-03-08 00:54:50 +01:00
Eugen Rochko
0a39c81dd8 Add test ensuring that unknown object types are rejected (#10166) 2019-03-05 11:46:36 +09:00
ThibG
833ffce2df Store remote votes URI (#10158)
* Store remote votes URI

* Add spec for accepting remote votes

* Make poll vote id generation work the same way as follows
2019-03-04 22:51:23 +01:00
Eugen Rochko
0e6998da3c
Add tests for ActivityPub poll processing (#10143) 2019-03-04 01:13:42 +01:00
Eugen Rochko
230a012f00
Add polls (#10111)
* Add polls

Fix #1629

* Add tests

* Fixes

* Change API for creating polls

* Use name instead of content for votes

* Remove poll validation for remote polls

* Add polls to public pages

* When updating the poll, update options just in case they were changed

* Fix public pages showing both poll and other media
2019-03-03 22:18:23 +01:00
ThibG
9d3c6f1849 Improved remote thread fetching (#10106)
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
2019-02-28 15:22:21 +01:00
Eugen Rochko
e7f20cc43f
Add type, limit, offset, min_id, max_id, account_id to search API (#10091)
* Add type, limit, offset, min_id, max_id, account_id to search API

Fix #8939

* Make the offset work on accounts and hashtags search as well

* Assure brakeman we are not doing mass assignment here

* Do not allow paginating unless a type is chosen

* Fix search query and index id field on statuses instead of created_at
2019-02-26 15:21:36 +01:00
Eugen Rochko
1a1b8170bb
Fix Announce activities of unknown statuses not fetching those statuses (#10065)
Regression from #9998
2019-02-17 15:16:36 +01:00
Eugen Rochko
147b4c2c3a
Add logging for rejected ActivityPub payloads and add tests (#10062) 2019-02-17 03:38:25 +01:00
Eugen Rochko
c417e8c198
Filter incoming Announce activities by relation to local activity (#10041)
* Filter incoming Announce activities by relation to local activity

Reject if announcer is not followed by local accounts, and is not
from an enabled relay, and the object is not a local status

Follow-up to #10005

* Fix tests
2019-02-15 18:19:45 +01:00
ThibG
6a5307a573 Alternative handling of private self-boosts (#9998)
* When self-boosting, embed original toot into Announce serialization

* Process unknown self-boosts from Announce object if it is more than an URI

* Add some self-boost specs

* Only serialize private toots in self-Announces
2019-02-13 18:36:23 +01:00
Franck Zoccolo
4f0322dcae Add support for IPv6 only MXes in Email validation (#10009)
* Add support for IPv6 only MXes

* Fixed email validator tests
2019-02-12 14:48:04 +01:00
Eugen Rochko
016ad37bc8
Fix URL linkifier grabbing full-width spaces and quotations (#9997)
Fix #9993
Fix #5654
2019-02-09 20:13:11 +01:00
Hinaloe
157d3af46c Only URLs extract with pre-escaped text (#9991)
* [test] add japanese hashtag testcase

* Only URLs extract with pre-escaped text

( https://github.com/tootsuite/mastodon/issues/9989 )
2019-02-09 03:39:38 +01:00
Eugen Rochko
364f2ff9aa
Add featured hashtags to profiles (#9755)
* Add hashtag filter to profiles

GET /@:username/tagged/:hashtag
GET /api/v1/accounts/:id/statuses?tagged=:hashtag

* Display featured hashtags on public profile

* Use separate model for featured tags

* Update featured hashtag counters on-write

* Limit featured tags to 10
2019-02-04 04:25:59 +01:00
Eugen Rochko
d14c276e58
Add option to overwrite imported data (#9962)
* Add option to overwrite imported data

Fix #7465

* Add import for domain blocks
2019-02-03 03:59:51 +01:00
Jakub Mendyk
6a5e3da6b0 Allow most kinds of characters in URL query (fixes #8408) (#8447)
* Allow unicode characters in URL query strings

Fixes #8408

* Alternative approach to unicode support in urls

Adds PoC/idea to approch this problem.
2019-02-02 19:01:18 +01:00
ThibG
e2a5be6e9a Prevent posting toots with media attachments from someone else (#9921) 2019-01-26 23:59:39 +01:00
ThibG
061feb63ed Fix scheduled toot with media immediately creating a toot (#9894)
* Add test for not persisting status when attaching media to scheduled toot

* Prevent status used for validation from being persisted to the database

Fixes #9893

Thanks to tateisu for the help investigating this.
2019-01-21 20:03:04 +01:00
ThibG
aeb124491d Reject existing Follow in addition to sending a Block (#9811)
Mastodon expects remote servers to remove follow relationships upon receiving
a Block. However, the spec only evokes Block activities in a C2S context, never
in a S2S context.

This PR, in addition to federating the Block, explicitly sends a Reject for any
affected follow relationship, which makes a bit more sense with regards to the
spec.
2019-01-18 15:57:19 +01:00
Eugen Rochko
bc642ac24b
Redesign public hashtag page to use a masonry layout (#9822) 2019-01-16 19:47:46 +01:00
Moritz Heiber
ecf40d09ed Disable Same-Site cookie implementation to fix SSO issues on WebKit browsers (#9819) 2019-01-15 23:11:46 +01:00
Renato "Lond" Cerqueira
5c5e14c816 Fix undefined method error in sidekiq (#9807)
* Fix undefined method error in sidekiq

Body can be not nil but still be empty, which causes a
`NoMethodError: undefined method `[]' for nil:NilClass` further in the
code. This checks for an empty body to avoid the issue.

* Fix codeclimate issue
2019-01-14 17:28:41 +01:00
ysksn
c059999ab3 Add a spec for Admin::ActionLog (#9775) 2019-01-11 07:28:09 +00:00
ysksn
09c3c96607 Add specs for Admin::AccountAction (#9767) 2019-01-11 07:26:03 +00:00
ysksn
61ecda1575 Not to skip executable specs (#9753)
* Not to skip executable specs

* Combine specs

Combine specs to one to reduce multiple slow http post.
2019-01-10 15:12:31 +01:00
Eugen Rochko
1c6588accc
Redesign admin instances area (#9645) 2019-01-08 13:39:49 +01:00
ysksn
9a38357111 Remove pending (#9752)
Some specs have already been added.
2019-01-08 09:42:56 +01:00
ysksn
274109e9f3 Remove spec files (#9751)
Nothing to test.
2019-01-08 12:18:46 +09:00
ysksn
88deca16ca Add pending specs for jsonld helper (#9750)
* Add specs for JsonLdHelper#first_of_value

* Add specs for JsonLdHelper#supported_context?
2019-01-08 12:18:27 +09:00
Eugen Rochko
a49d43d112
Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
ysksn
5efedb5d5e Add specs for UrlValidator (#9699) 2019-01-03 13:10:20 +09:00
ysksn
19abf4ef0b Add specs for UnreservedUsernameValidator (#9698)
* Add specs for UnreservedUsernameValidator

* Use instance variable
2019-01-03 13:10:02 +09:00
Eugen Rochko
66436d0895
Improve e-mail digest (#9689)
- Reduce time-to-digest from 20 to 7 days
- Fetch mentions starting from +1 day since last login
- Fix case when last login is more recent than last e-mail
- Do not render all mentions, only 40, but show number in subject
- Do not send digest to moved accounts
- Do send digest to silenced accounts
2019-01-02 10:47:32 +01:00
ThibG
70be301d69 Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order (#9687)
* Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order

* Add specs for Block causing unfollow and for out-of-order Block + Undo
2019-01-02 01:12:02 +01:00
ThibG
290932602b Reduce usage of LD signatures (#9659)
* Do not LDS-sign Follow, Accept, Reject, Undo, Block

* Do not use LDS for Create activities of private toots

* Minor cleanup

* Ignore unsigned activities instead of misattributing them

* Use status.distributable? instead of querying visibility directly
2018-12-30 09:48:59 +01:00
ysksn
fb08039de5 Add specs for FollowLimitValidator (#9655) 2018-12-29 08:24:52 +01:00
ysksn
05edec6917 Add specs for BlackListedEmailValidator (#9651)
* Add specs for BlackListedEmailValidator

* Use instance variable
2018-12-29 07:23:44 +01:00
ysksn
4725aeec9f Add specs for DisallowedHashtagsValidator (#9653)
In order to implement tests easier, `#select_tags` created.
2018-12-29 07:22:51 +01:00
Eugen Rochko
0f938ff29c
Add handler for Move activity (#9629) 2018-12-29 02:24:36 +01:00
ysksn
d01c840e14 Add specs for StatusPinValidator (#9648) 2018-12-28 18:09:32 +09:00
ysksn
ccb9c1b952 Add pending specs for StatusLengthValidator (#9647)
* Add pending specs of StatusLengthValidator

* Use instance variable
2018-12-28 08:18:47 +01:00
Eugen Rochko
5d2fc6de32
Add REST API for creating an account (#9572)
* Add REST API for creating an account

The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.

The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.

The method is rate-limited by IP to 5 requests per 30 minutes.

* Redirect users back to app from confirmation if they were created with an app

* Add tests

* Return 403 on the method if registrations are not open

* Require agreement param to be true in the API when creating an account
2018-12-24 19:12:38 +01:00
ThibG
5f387995d9 Limit maximum visibility of local silenced users to unlisted (#9583)
Fixes #9580
2018-12-24 19:06:14 +01:00
Eugen Rochko
3c033c4352
Add moderation warnings (#9519)
* Add moderation warnings

Replace individual routes for disabling, silencing, and suspending
a user, as well as the report update route, with a unified account
action controller that allows you to select an action (none,
disable, silence, suspend) as well as whether it should generate an
e-mail notification with optional custom text. That notification,
with the optional custom text, is saved as a warning.

Additionally, there are warning presets you can configure to save
time when performing the above.

* Use Account#local_username_and_domain
2018-12-22 20:02:09 +01:00
ysksn
eee2b05ea2 Add specs for CustomEmojiFilter (#9599) 2018-12-21 18:52:57 +01:00
ysksn
de3cecf37a Add specs for AdminMailer (#9597) 2018-12-21 09:34:34 +01:00
ysksn
1bc78ec50e Add specs for InstancePresenter (#9596) 2018-12-21 08:59:56 +01:00
ysksn
6a2d030c2f Add specs for ReportNotePolicy (#9592) 2018-12-20 17:52:18 +01:00
ysksn
b93e317886 Add specs for policies (#9591)
* Add spec for RelayPolicy

* Add specs for SubscriptionPolicy

* Add specs for SettingsPolicy

* Add specs for TagPolicy

* Add specs for ReportPolicy
2018-12-20 17:52:07 +01:00
ysksn
be9640bfc2 Add specs for UserPolicy (#9593) 2018-12-20 17:51:55 +01:00
ysksn
d649d84594 Add specs for InvitePolicy (#9589) 2018-12-20 04:23:09 +01:00
ysksn
44189c33d1 Add specs for EmailDomainBlockPolicy (#9586) 2018-12-20 03:51:41 +01:00
ysksn
08cb8a1ff3 Add specs for InstancePolicy (#9587) 2018-12-20 03:51:31 +01:00
ysksn
e181f99739 Add specs for DomainBlockPolicy (#9585) 2018-12-20 03:47:51 +01:00
ysksn
5088213f5e Add specs for CustomEmojiPolicy (#9584) 2018-12-20 03:24:28 +01:00
ysksn
5d724aa129 Add specs for BackupPolicy (#9576) 2018-12-19 18:24:15 +01:00
ysksn
af56efdec5 Add specs for AccountPolicy (#9575) 2018-12-19 08:56:59 +01:00
ysksn
0a1ade4f02 Add specs for AccountModerationNotePolicy (#9571) 2018-12-19 07:24:03 +01:00
ysksn
102e4cfa32 Add specs for StatusPolicy (#9569) 2018-12-19 05:19:20 +01:00
ysksn
dd85700a3e Add spec for AccountableConcern#log_action (#9559) 2018-12-18 16:43:03 +01:00
ThibG
e709b8da0d Ignore low-confidence CharlockHolmes guesses when parsing link cards (#9510)
* Add failing test for windows-1251 link cards

* Ignore low-confidence CharlockHolmes guesses

Fixes #9466

* Fix no method error when charlock holmes cannot detect charset
2018-12-17 19:19:45 +01:00
ysksn
0c80715235 Add spec for Api::V1::Timelines::DirectController (#9547) 2018-12-17 11:36:20 +01:00
ysksn
351938520d Add specs for Api::V1::Instances::PeersController (#9546) 2018-12-17 11:35:55 +01:00
ysksn
2d871feb10 Add spec for Api::V1::EndorsementsController (#9543) 2018-12-17 11:32:44 +01:00
ysksn
3fa9615cb3 Add spec for Api::V1::Instances::ActivityController (#9545) 2018-12-17 11:32:24 +01:00
ysksn
a3dcbfddd6 Add specs for Accounts::PinsController (#9542) 2018-12-17 06:03:51 +01:00
ysksn
3c31c28605 Add spec for Admin::ActionLogsController#index (#9522) 2018-12-14 20:37:01 +01:00
ysksn
458e2b0c5b Add specs for RemoteInteractionController (#9524) 2018-12-14 20:36:40 +01:00
ysksn
c1600a0f69 Add spec for Admin::DashboardController#index (#9523) 2018-12-14 20:36:18 +01:00
Sumit Khanna
769c2d2680 Error message for avatar image that's too large. #9204 (#9518)
* Error message for avatar image that's too large. #9204

* Code climate/formatting

* Removed avatar error message

* Moved valid image dimentions check to update service

* removed unnescessary begin block

* code climate formatting

* code climate indent fix
2018-12-14 05:07:21 +01:00
ysksn
795bac44fd Add spec for Settings::ExportsController#create (#9512) 2018-12-13 02:53:52 +01:00
Adam Copp
7d00e4edbd Make custom emoji domains case insensitive #9351 (#9474)
* Make custom emoji domains case sensitive #9351

* Fixup style in downcase_domain to comply with codeclimate.

* switch if! to unless

* Don't use transactions, operate in batches.

Also revert spurious schema change.
2018-12-11 05:30:57 +01:00
Eugen Rochko
dbb1ee269f
Improve e-mail MX validator and add tests (#9489) 2018-12-10 22:53:25 +01:00
ysksn
ed24bb2c3e Add specs for activitypub collections controller (#9484)
* Add specs for ActivityPub::CollectionsController#show

* Raise ActiveRecord::RecordNotFound

Raising ActiveRecord::NotFound raises NameError: uninitialized constant
ActiveRecord::NotFound.
2018-12-10 21:39:25 +01:00
ysksn
6eae8f77af Add spec for Admin::SuspentionsController#new (#9483) 2018-12-10 21:38:21 +01:00
ysksn
361818e931 Fix Admin::TagsController#unhide (#9481) 2018-12-10 21:37:38 +01:00
ysksn
ae3d2f446a Add specs for Admin::InvitesController (#9471) 2018-12-10 01:19:28 +09:00
ysksn
dfd123d5b3 Remove pending spec (#9453) 2018-12-07 16:53:55 +01:00
ysksn
d3547fa005 Add specs for ActivityPub::InboxesController (#9456) 2018-12-07 16:40:01 +01:00
ysksn
88b3eed16f Add specs for Admin::AccountModerationNotesHelper (#9455) 2018-12-07 16:39:20 +01:00
ysksn
57bb62d5cf Remove pending spec (#9454)
Since dots are not allowed in username,
this spec is no longer needed.
2018-12-07 16:38:50 +01:00
ysksn
51cbd045da Add specs for AccountTagStat model (#9452) 2018-12-07 16:37:56 +01:00
Eugen Rochko
73be8f38c1
Add profile directory (#9427)
Fix #5578
2018-12-06 17:36:11 +01:00
ysksn
155cf12680 Remove pending spec (#9442)
`#from_account` isn't defined.
2018-12-06 17:39:15 +09:00
ysksn
e2910dff12 Add spec for Identity.find_for_oauth (#9441) 2018-12-06 17:38:49 +09:00
ThibG
e88c6a5c3c Fix thread depth computation in statuses_controller (#9426)
* Add test that should currently fail

* Fix depth computation (will still fail if statuses have been filtered out)

* Fix handling of broken threads
2018-12-05 02:12:29 +01:00
ThibG
395615d9f3 Allow hyphens in the middle of remote user names (#9345)
Fixes #9309

This only allows hyphens in the middle of a username, much like dots,
although I don't have a compelling reason to do so other than keeping
the changes minimal.
2018-11-27 12:28:01 +01:00
Eugen Rochko
73faadad28
Redesign admin accounts index (#9340)
* Improve overview of accounts in admin UI

- Display suspended status, role, last activity and IP prominently
- Default to showing local accounts
- Default to not showing suspended accounts

* Remove unused strings

* Fix tests

* Allow filtering accounts by IP mask
2018-11-26 15:53:27 +01:00
valerauko
db9aea34de Ensure replied-to is a status not a boost (#9129)
* Ensure replied-to is a status not a boost

* Consider case of not a reply

* Add test case for replying to boost

* Move reblog-reply resolution to model

* Remove unnecessary comment
2018-11-25 16:35:21 +01:00
Eugen Rochko
0eaf6d7693
Sort self-replies to the top of descendants (#9320)
Fix #6463
2018-11-24 20:48:50 +01:00
Eugen Rochko
fd8145d232
Fix connect timeout not being enforced (#9329)
* Fix connect timeout not being enforced

The loop was catching the timeout exception that should stop execution, so the next IP would no longer be within a timed block, which led to requests taking much longer than 10 seconds.

* Use timeout on each IP attempt, but limit to 2 attempts

* Fix code style issue

* Do not break Request#perform if no block given

* Update method stub in spec for Request

* Move timeout inside the begin/rescue block

* Use Resolv::DNS with timeout of 1 to get IP addresses

* Update Request spec to stub Resolv::DNS instead of Addrinfo

* Fix Resolve::DNS stubs in Request spec
2018-11-22 20:12:04 +01:00
valerauko
824497fbce Ignore JSON-LD profile in mime type comparison (#9179)
Ignore JSON-LD profile in mime type comparison
2018-11-22 12:49:07 +01:00
Eugen Rochko
d6b9a62e0a
Extract counters from accounts table to account_stats table (#9295) 2018-11-19 00:43:52 +01:00
Eugen Rochko
8069fd636b
Remove intermediary arrays when creating hash maps from results (#9291) 2018-11-16 15:02:18 +01:00
Eugen Rochko
6d59dfa15d
Optimize the process of following someone (#9220)
* Eliminate extra accounts select query from FollowService

* Optimistically update follow state in web UI and hide loading bar

Fix #6205

* Asynchronize NotifyService in FollowService

And fix failing test

* Skip Webfinger resolve routine when called from FollowService if possible

If an account is ActivityPub, then webfinger re-resolving is not necessary
when called from FollowService. Improve options of ResolveAccountService
2018-11-08 21:05:42 +01:00
James Kiesel
4c03e05a4e Allow joining several hashtags in a single column (#8904)
* Nascent tag menu on frontend

* Hook up frontend to search

* Tag intersection backend first pass

* Update yarnlock

* WIP

* Fix for tags not searching correctly

* Make radio buttons function

* Simplify radio buttons with modeOption

* Better naming

* Rearrange options

* Add all/any/none functionality on backend

* Small PR cleanup

* Move to service from scope

* Small cleanup, add proper service tests

* Don't use send with user input :D

* Set appropriate column header

* Handle auto updating timeline

* Fix up toggle function

* Use tag value correctly

* A bit more correct to use 'self' rather than 'all' in status scope

* Fix some style issues

* Fix more code style issues

* Style select dropdown more better

* Only use to_id'ed value to ensure no SQL injection

* Revamp frontend to allow for multiple selects

* Update backend / col header to account for more flexible tagging

* Update brakeman ignore

* Codeclimate suggestions

* Fix presenter tag_url

* Implement initial PR feedback

* Handle additional tag streaming

* CodeClimate tweak
2018-11-05 18:53:25 +01:00
Eugen Rochko
ce2ee68b64
Revert "Fix FetchAtomService content type handling (#9132)" (#9171)
This reverts commit c36a4a1617.
2018-10-31 00:43:34 +01:00
valerauko
c36a4a1617 Fix FetchAtomService content type handling (#9132)
* Add profile to json+ld in Accept

It's required by the ActivityPub spec

* Use headers['Content-type'] instead of mime_type

mime_type strips the profile from the content type, but it's still available raw in the headers hash

* Add test for ld+json with profile
2018-10-30 15:07:57 +01:00
ThibG
33a71e8f7c Do not hide boost notifications from followed people with hidden boosts (#9147)
* Do not hide boost notifications from followed people with hidden boosts

Not displaying boosts from a followed user in the Home timeline and not
having notifications when they reblog your own content are two very
separate concerns, tying them together seem counter-intuitive and unwanted.

* Update specs accordingly
2018-10-30 00:47:31 +01:00
Eugen Rochko
b40ea6d1d4
Bump sanitize from 4.6.6 to 5.0.0 (#9140) 2018-10-29 14:05:25 +01:00
takayamaki
33976c8ecc fix: Execute PAM authentication tests on CircleCI (#9029)
and use 'if' option of context block
2018-10-20 17:28:04 +02:00
Eugen Rochko
d5bfba3262
Do not test PAM authentication by default (#9027)
* Do not test PAM authentication by default

* Disable PAM tests if PAM is not enabled
2018-10-20 07:32:26 +02:00
Eugen Rochko
ddd30f331c
Improve support for aspects/circles (#8950)
* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification
2018-10-17 17:13:04 +02:00
Eugen Rochko
21ad21cb50
Improve signature verification safeguards (#8959)
* Downcase signed_headers string before building the signed string

The HTTP Signatures draft does not mandate the “headers” field to be downcased,
but mandates the header field names to be downcased in the signed string, which
means that prior to this patch, Mastodon could fail to process signatures from
some compliant clients. It also means that it would not actually check the
Digest of non-compliant clients that wouldn't use a lowercased Digest field
name.

Thankfully, I don't know of any such client.

* Revert "Remove dead code (#8919)"

This reverts commit a00ce8c92c.

* Restore time window checking, change it to 12 hours

By checking the Date header, we can prevent replaying old vulnerable
signatures. The focus is to prevent replaying old vulnerable requests
from software that has been fixed in the meantime, so a somewhat long
window should be fine and accounts for timezone misconfiguration.

* Escape users' URLs when formatting them

Fixes possible HTML injection

* Escape all string interpolations in Formatter class

Slightly improve performance by reducing class allocations
from repeated Formatter#encode calls

* Fix code style issues
2018-10-12 00:15:55 +02:00
ashleyhull-versent
f194857ac9 rubocop issues - Cleaning up (#8912)
* cleanup pass

* undo mistakes

* fixed.

* revert
2018-10-08 04:50:11 +02:00
ashleyhull-versent
2dba313100 Replace SVG asset with Custom mascot (#8766) 2018-10-08 00:20:45 +02:00
Eugen Rochko
774ac47373
Add conversations API (#8832)
* Add conversations API

* Add web UI for conversations

* Add test for conversations API

* Add tests for ConversationAccount

* Improve web UI

* Rename ConversationAccount to AccountConversation

* Remove conversations on block and mute

* Change last_status_id to be a denormalization of status_ids

* Add optimistic locking
2018-10-07 23:44:58 +02:00
Jeong Arm
144d73730d Leave unknown language as nil if account is remote (#8861)
* Force use language detector if account is remote

* Set unknown remote toot's language as nil
2018-10-05 19:17:46 +02:00
aus-social
0a4739c732 lint pass 2 (#8878)
* Code quality pass

* Typofix

* Update applications_controller_spec.rb

* Update applications_controller_spec.rb
2018-10-04 17:38:04 +02:00
Eugen Rochko
a46ab86adf
Limit the number of people that can be followed from one account (#8807)
Configurable soft limit of 7,500, and above that, configurable
ratio of 1.1 * followers, controlled by:

- MAX_FOLLOWS_THRESHOLD
- MAX_FOLLOWS_RATIO

Fix #2311
2018-10-04 17:36:11 +02:00
Eugen Rochko
e645ae9561
Change admin accounts default sort to most recent (#8813) 2018-10-04 16:05:38 +02:00
Eugen Rochko
7fe137d2f7
Fix link verification for remote accounts (#8868) 2018-10-04 15:47:03 +02:00
aus-social
1f98eae1cf Lint pass (#8876) 2018-10-04 12:36:53 +02:00
Eugen Rochko
f0fff3eb10
Support min_id-based pagination in REST API (#8736)
* Allow min_id pagination in Feed#get

* Add min_id pagination to home and list timeline APIs

* Add min_id pagination to account statuses, public and tag APIs

* Remove unused stub in reports API

* Use min_id pagination in notifications, favourites, and fix order

* Fix HomeFeed#from_database not using paginate_by_id
2018-09-28 02:23:45 +02:00
Eugen Rochko
3d7f68c273
Revert Font Awesome 5 upgrade (#8810)
* Revert "Fix some icon names changed by the Font Awesome 5. (#8796)"

This reverts commit 3f9ec3de82.

* Revert "Migrate to font-awesome 5.0. (#8799)"

This reverts commit 8bae14591b.

* Revert "Fix some icons names, unavailable in fontawesome5 (free license). (#8792)"

This reverts commit b9c727a945.

* Revert "Update the icon name changed by the Font Awesome 5. (#8776)"

This reverts commit 17af4d27da.

* Revert "Add bot icon to bot avatars and migrate to newer version of Font Awesome (#8484)"

This reverts commit 4b794e134d.
2018-09-28 02:11:14 +02:00
Naoki Kosaka
8bae14591b Migrate to font-awesome 5.0. (#8799) 2018-09-27 17:08:56 +02:00
ThibG
c39183cc62 Refactor active_nav_class for use with multiple paths (#8757) 2018-09-23 20:42:13 +02:00
Eugen Rochko
f92f1ee80a
Support link verification with redirects (#8735)
(e.g. URL shortener)
2018-09-20 00:10:35 +02:00
Yamagishi Kazutoshi
3da1cc7d5e Fix failed profile verification when rel attribute including values other than me (#8733) 2018-09-19 16:47:31 +02:00
Eugen Rochko
f4d549d300
Redesign forms, verify link ownership with rel="me" (#8703)
* Verify link ownership with rel="me"

* Add explanation about verification to UI

* Perform link verifications

* Add click-to-copy widget for verification HTML

* Redesign edit profile page

* Redesign forms

* Improve responsive design of settings pages

* Restore landing page sign-up form

* Fix typo

* Support <link> tags, add spec

* Fix links not being verified on first discovery and passive updates
2018-09-18 16:45:58 +02:00
luzpaz
40dd19be37 Misc. typos (#8694)
Found via `codespell -q 3 --skip="./app/javascript/mastodon/locales,./config/locales"`
2018-09-14 00:53:09 +02:00
Eugen Rochko
2288d50a7b
Add force_login option to OAuth authorize page (#8655)
* Add force_login option to OAuth authorize page

For when a user needs to sign into an app from multiple accounts
on the same server

* When logging out from modal header, redirect back after re-login
2018-09-09 04:10:44 +02:00
Sorin Davidoi
6f3d934bc1 feat(cookies): Use the same-site attribute to lax (#8626)
CSFR-prevention is already implemented but adding this doesn't hurt.

A brief introduction to Same-Site cookies (and the difference between strict and
lax) can be found at
https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/

TLDR: We use lax since we want the cookies to be sent when the user navigates
safely from an external site.
2018-09-08 23:54:28 +02:00
Eugen Rochko
c593d6df9c
Add preference for report notification e-mails, skip for duplicates (#8559)
If an unresolved report for the same target account already exists,
no new notification is generated
2018-09-02 00:11:58 +02:00
Renato "Lond" Cerqueira
fe56d26f7b Fix autoplay issue with spoiler tag (#8540)
Add tests to avoid similar issues in the future
2018-08-31 15:16:59 +02:00
Renato "Lond" Cerqueira
11658d8653 Add animate custom emoji param to embed pages (#8507)
* Add animate custom emoji param to embed pages

* Rename param, use it for avatars and gifs

* Fix issues pointed by codeclimate and breaking test

* Ignore brakeman warning
2018-08-30 23:14:01 +02:00
Renato "Lond" Cerqueira
5b2b493a90 Fix nil host in remotable (#8508)
Host can be nil in urls like
'https:https://example.com/path/file.png'
2018-08-29 21:13:49 +02:00
sundevour
4bfd786550 formatter spec fixes & clarification (#8481)
updates some "context" and "it" lines to have clearer explanations
updates "context" lines to properly describe function input, and "it" lines to describe results
2018-08-29 01:20:56 +02:00
Eugen Rochko
5e1767173f
Display pending message on admin relays UI (#8494)
* Add missing specs for relay accept/reject

* Display pending message on admin relays UI
2018-08-28 05:39:43 +02:00
Jakub Mendyk
f3a12ddfd0 Make Api::V1::MutesController paginate properly (#8472)
Fixes #8463
2018-08-26 21:30:17 +02:00
Eugen Rochko
22e46ebad8
Add theme identifier to body classes for easier custom CSS styling (#8439)
Add forgotten custom CSS admin setting strings
2018-08-25 22:55:25 +02:00
Eugen Rochko
2f34b747b3
Allow mods to disable login, improve message when login disabled (#8329)
* Allow moderators to disable/enable login

* Instead of rejecting login, show forbidden error when login disabled

Avoid confusion because when login is rejected, the message is that
the account is not activated, which is wrong.

* Fix tests
2018-08-23 23:26:29 +02:00
Jakub Mendyk
6cb3514d64 Add ability to change an instance default theme from the administration panel (#7092) (#8381)
* Add default_settings class method to ScopedSettings

ScopedSettings was extended to use value of unscoped setting instead of
only using defaults set in config/settings.yml for selected settings.
This adds possibility for admins to set default values of users' settings,
for example default theme (as requested in #7092).

* Add ability to change an instance default theme

Closes #7092
2018-08-23 14:17:35 +02:00
Eugen Rochko
802cf6a4c5
Improve federated ID validation (#8372)
* Fix URI not being sufficiently validated with prefetched JSON

* Add additional id validation to OStatus documents, when possible
2018-08-22 20:55:14 +02:00
masarakki
4bdab203ac exclude-other-silenced-accounts (#7528) 2018-08-22 13:20:50 +02:00
Eugen Rochko
2374a00c10
Add confirmation step to account suspensions (#8353)
* Add confirmation page for suspensions

* Suspension confirmation closes reports, linked from report UI

* Fix tests
2018-08-22 11:53:41 +02:00
Eugen Rochko
6226aa83d7
Increase reach of Delete->Actor activities (#8305)
Fix #7316
2018-08-20 13:28:05 +02:00
Eugen Rochko
d010816ba8
Fix error when trying to update counters for statuses that are gone (#8251) 2018-08-18 03:03:23 +02:00
Eugen Rochko
78fa926ed5
Add remote interaction dialog for toots (#8202)
* Add remote interaction dialog for toots

* Change AuthorizeFollow into AuthorizeInteraction, support statuses

* Update brakeman.ignore

* Adjust how interaction buttons are display on public pages

* Fix tests
2018-08-18 03:03:12 +02:00
ThibG
59f7f4c923 Implement Undo { Accept { Follow } } (fixes #8234) (#8245)
* Add Follow#revoke_request!

* Implement Undo { Accept { Follow } } (fixes #8234)
2018-08-17 16:24:56 +02:00
ThibG
af912fb308 Allow accessing local private/DM messages by URL (#8196)
* Allow accessing local private/DM messages by URL

(Provided the user pasting the URL is authorized to see the toot, obviously)

* Fix SearchServiceSpec tests
2018-08-15 19:33:36 +02:00
Eugen Rochko
aaac14b8ad
Show exact number of followers/statuses on export page/in tooltip (#8199)
* Show exact number of followers/statuses on export page/in tooltip

* Fix tests
2018-08-14 21:56:17 +02:00
Eugen Rochko
8e111b753a
Move status counters to separate table, count replies (#8104)
* Move status counters to separate table, count replies

* Migration to remove old counter columns from statuses table

* Fix schema file
2018-08-14 19:19:32 +02:00
S.H
2aeeffc3ec Update Rails (#8141)
* Update Rails

* fix Update Rails
2018-08-12 12:25:23 +02:00
Eugen Rochko
0dcc1950d1
Update /terms and /about/more to use public layout (#8142) 2018-08-09 12:58:20 +02:00
Eugen Rochko
f2404de871
Public profile endorsements (accounts picked by profile owner) (#8146) 2018-08-09 09:56:53 +02:00
Eugen Rochko
cc56f2230a
Add separate setting for sidebar text (site_short_description) (#8107)
* Add separate setting for sidebar text (site_short_description)

* Fix tests
2018-07-31 18:59:34 +02:00
Eugen Rochko
60df87f6f0
Compensate for scrollbar disappearing when media modal visible (#8100)
* Compensate for scrollbar disappearing when media modal visible

Make auth pages backgrounds lighter

* Fix typo
2018-07-31 01:14:33 +02:00
Eugen Rochko
e7e577dd6e
Enforce username format for remote users, too (#8102)
Initially I thought there might be valid reasons for remote users to
have a different, unpredicted username format. However, I now realize
such a difference would be unusable and unexpected within Mastodon.

Fix #8058
2018-07-30 22:29:52 +02:00
Eugen Rochko
bb71538bb5
Redesign public profiles and toots (#8068) 2018-07-28 19:25:33 +02:00
Eugen Rochko
0fb0037ca7
Resize images by area instead of fixed dimensions (#8083)
To improve the way super tall or super ride images are treated, the
numbers remain the same, 1280x1280 and 400x400, but if an image
is less in one dimension than the other, the other can become larger

Thanks to @WAHa_06x36@mastodon.social for the tip
2018-07-28 03:33:00 +02:00
Eugen Rochko
2a176514be
Display full acct on public status pages, always (#8011) 2018-07-14 04:07:47 +02:00
Eugen Rochko
38e9662d78
Disable language detection for texts shorter than 140 characters (#8010)
If the input text is blank after preparation (only mention, or
only URL, or empty as in a media post), then use nil as language,
since it's OK to show to everyone.

Otherwise, always fall back to the server's default locale
2018-07-14 04:05:36 +02:00
Eugen Rochko
e55dce3176
Add federation relay support (#7998)
* Add federation relay support

* Add admin UI for managing relays

* Include actor on relay-related activities

* Fix i18n
2018-07-13 02:16:06 +02:00
ThibG
1ca4e51eb3 Add option to not consider word boundaries when processing keyword filtering (#7975)
* Add option to not consider word boundaries when filtering phrases

* Add a few tests for keyword/phrase filtering
2018-07-09 02:22:09 +02:00
Eugen Rochko
cd509d2146
Remove .p-name microformat class (#7961)
Fix #7926
2018-07-07 18:51:56 +02:00
Eugen Rochko
1f6ed4f86a
Add more granular OAuth scopes (#7929)
* Add more granular OAuth scopes

* Add human-readable descriptions of the new scopes

* Ensure new scopes look good on the app UI

* Add tests

* Group scopes in screen and color-code dangerous ones

* Fix wrong extra scope
2018-07-05 18:31:35 +02:00
Eugen Rochko
da8fe8079e
Re-add follow recommendations API (#7918)
* Re-add follow recommendations API

    GET /api/v1/suggestions

Removed in 8efa081f21 due to Neo4J
dependency. The algorithm uses triadic closures, takes into account
suspensions, blocks, mutes, domain blocks, excludes locked and moved
accounts, and prefers more recently updated accounts.

* Track interactions with people you don't follow

Replying to, favouriting and reblogging someone you're not following
will make them show up in follow recommendations. The interactions
have different weights:

- Replying is 1
- Favouriting is 10 (decidedly positive interaction, but private)
- Reblogging is 20

Following them, muting or blocking will remove them from the list,
obviously.

* Remove triadic closures, ensure potential friendships are trimmed
2018-07-03 01:47:56 +02:00
Eugen Rochko
cdb101340a
Keyword/phrase filtering (#7905)
* Add keyword filtering

    GET|POST       /api/v1/filters
    GET|PUT|DELETE /api/v1/filters/:id

- Irreversible filters can drop toots from home or notifications
- Other filters can hide toots through the client app
- Filters use a phrase valid in particular contexts, expiration

* Make sure expired filters don't get applied client-side

* Add missing API methods

* Remove "regex filter" from column settings

* Add tests

* Add test for FeedManager

* Add CustomFilter test

* Add UI for managing filters

* Add streaming API event to allow syncing filters

* Fix tests
2018-06-29 15:34:36 +02:00
Shuhei Kitagawa
23955d956e Add tests for remote_unfollows_controller (#7879) 2018-06-24 19:55:55 +09:00
Shuhei Kitagawa
459394a020 Add missing tests for confirmations controller (#7866) 2018-06-21 10:40:23 +09:00
Shuhei Kitagawa
63b05096c7 Add tests for shares_controller (#7835) 2018-06-18 10:45:20 +09:00
Eugen Rochko
7eec279c7f
Change language opt-out to language opt-in (#7823)
* Switch filtered_languages to chosen_languages

* Adjust interface

* Remove unused translations
2018-06-17 13:54:02 +02:00
Shuhei Kitagawa
ad8814232f Add tests for following accounts controller (#7800) 2018-06-14 10:49:17 +09:00
Shuhei Kitagawa
5b47774ab8 Add tests for followers_accounts_controller (#7794) 2018-06-13 10:28:39 +09:00
Shuhei Kitagawa
6151308c47 Add missing tests for admin/accounts_controller (#7791) 2018-06-12 21:24:46 +09:00
Shuhei Kitagawa
7086aa598b Add tests for intents_controller (#7763) 2018-06-09 22:47:50 +02:00
Eugen Rochko
10f51c9886
Fix domain hiding logic (#7765)
* Send rejections to followers when user hides domain they're on

* Use account domain blocks for "authorized followers" action

Replace soft-blocking (block & unblock) behaviour with follow rejection

* Split sync and async work of account domain blocking

Do not create domain block when removing followers by domain, that
is probably unexpected from the user's perspective.

* Adjust confirmation message for domain block

* yarn manage:translations
2018-06-09 22:46:54 +02:00
Shuhei Kitagawa
12fa2500c4 Add missing tests for sessions controller (#7744) 2018-06-06 10:23:22 +09:00
Shuhei Kitagawa
22caa32ba2 Add tests for embeds controller (#7719)
* Small refactoring of status_finder_spec

* Add tests for embeds_controller
2018-06-04 10:35:56 +09:00
Shuhei Kitagawa
00512ecf87 Add tests for migrations_controller (#7707) 2018-06-02 18:52:16 +09:00
Eugen Rochko
a7d726c383
Improve counter caches on Status and Account (#7644)
Do not touch statuses_count on accounts table when mass-destroying
statuses to reduce load when removing accounts, same for
reblogs_count and favourites_count

Do not count statuses with direct visibility in statuses_count

Fix #828
2018-05-30 02:50:23 +02:00
Shuhei Kitagawa
b0b34a5e38 Add a test for emojis_controller (#7652) 2018-05-28 22:56:58 +09:00
tateisu
b87a1229c7 optimize direct timeline (#7614)
* optimize direct timeline

* fix typo in class name

* change filter condition for direct timeline

* fix codestyle issue

* revoke index_accounts_not_silenced because direct timeline does not use it.

* revoke index_accounts_not_silenced because direct timeline does not use it.

* fix rspec test condition.

* fix rspec test condition.

* fix rspec test condition.

* revoke adding column and partial index

* (direct timeline) move merging logic to model

* fix pagination parameter

* add method arguments that switches return array of status or cache_ids

* fix order by

* returns ActiveRecord.Relation in default behavor

* fix codestyle issue
2018-05-28 11:04:06 +02:00
Yamagishi Kazutoshi
6d99a0b652 Fix tests for invites controller (regression from 4d81809f36) (#7597) 2018-05-23 06:32:10 +02:00
Shuhei Kitagawa
12e590edd7 Add tests for report notes controller (#7589) 2018-05-22 14:45:10 +02:00
Eugen Rochko
8378b72eba
Ensure push subscription is immediately removed when application is revoked (#7548)
* Ensure push subscription is immediately removed when application is revoked

* When token is revoked from app, unsubscribe too
2018-05-19 21:05:08 +02:00
Eugen Rochko
57b503d4ef
Resolve unknown status from Add activity, skip Remove if unknown (#7526)
Fix #7518
2018-05-18 11:33:56 +02:00
Shuhei Kitagawa
b48a166c82 Add tests for account_moderation_notes_controller (#7524) 2018-05-17 04:26:51 +02:00
Shuhei Kitagawa
50491e0d92 Add tests for invites controller (#7441)
* Add tests for invites controller

* Small refactoring and fix for invites controller
2018-05-11 13:14:33 +02:00
Eugen Rochko
b4fb766b23
Add REST API for Web Push Notifications subscriptions (#7445)
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
2018-05-11 11:49:12 +02:00
Yamagishi Kazutoshi
ac788ad47e Disable simplecov on CircleCI (#7416)
* Disable simplecov on CircleCI

* Remove --format progress
2018-05-09 16:59:58 +02:00
Shuhei Kitagawa
ce35d81db7 Add tests for admin/roles_controller (#7421) 2018-05-09 08:41:46 +02:00
Shuhei Kitagawa
35eff337d5 Add tests for admin/invites_controller (#7412) 2018-05-09 08:41:26 +02:00
Surinna Curtis
01dfd6dbc8 Take the first recognized actor_type. (#7410) 2018-05-08 13:30:04 +02:00
Shuhei Kitagawa
bd10a7e480 Add resend confirmation for admin (#7378)
* Add confirmations_controller#resend

* Add tests for confirmations_controller#resend

* Add translations
2018-05-06 10:59:03 +02:00
Yamagishi Kazutoshi
8c35d163a5 Port travis_retry for CI (#7379)
* Port travis_retry for CI

* Add license
2018-05-06 10:55:50 +02:00
Eugen Rochko
c7d1a2e400
Improve admin UI for accounts (#7360)
* Improve design of account statuses admin UI (consistent with reports)

* Make account moderation notes look consistent with report notes

* i18n-tasks remove-unused

* Fix code style issues

* Fix tests
2018-05-05 23:06:29 +02:00
Shuhei Kitagawa
661f7e6d9d Add tests for admin/custom_emojis_controller (#7350) 2018-05-05 15:53:59 +02:00
Eugen Rochko
c73ce7b695
Store home feeds for 7 days instead of 14 (#7354)
* Store home feeds for 7 days instead of 14

Reduces workload for status fan-out to active followers

* Fix test for user model
2018-05-05 00:54:24 +02:00
Eugen Rochko
6793bec4c6
Store URIs of follows, follow requests and blocks for ActivityPub (#7160)
Same URI passed between follow request and follow, since they are
the same thing in ActivityPub. Local URIs are generated during
creation using UUIDs and are passed to serializers.
2018-05-04 21:14:34 +02:00
Shuhei Kitagawa
28b366d065 Add missing tests for report.rb (#7324) 2018-05-03 10:42:36 +02:00
Eugen Rochko
28bd4b9800
Serialize webfinger XML with Ox instead of Nokogiri (#7319)
25ms -> 0.5ms
2018-05-02 22:28:46 +02:00
Eugen Rochko
cb5b5cb5f7
Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
abcang
71a7cea73f Keep notification when muting_notifications is true (#7311)
* Keep notification when muting_notifications is true

* Retrun mute object

* Fix test
2018-05-02 16:14:51 +02:00
Eugen Rochko
c5dcd7d836
Speed up test suite by not generating RSA keys in test environment (#7296)
One RSA keypair for all fabricated test accounts is enough
2018-05-02 15:45:24 +02:00
Shuhei Kitagawa
6c40e567aa Add missing tests for user.rb (#7306) 2018-05-02 14:13:52 +02:00
Surinna Curtis
dc786c0cf4 Support Actors/Statuses with multiple types (#7305)
* Add equals_or_includes_any? helper in JsonLdHelper

* Support arrays in JSON-LD type fields for actors/tags/objects.

* Spec for resolving accounts with extension types

* Style tweaks for codeclimate
2018-05-02 12:40:24 +02:00
MIYAGI Hikaru
eb593a5a0c Append '.test' to hostname in stub data (#7260) 2018-04-25 14:12:28 +02:00
Emelia Smith
495303d9b8 Prevent suspended accounts from appearing in AccountSearchService (#7246) 2018-04-23 21:27:18 +02:00
Akihiko Odaki
1258efa882 Paginate descendant statuses in public page (#7148) 2018-04-23 19:27:35 +02:00
Eugen Rochko
75c4ab9d12
Remove "nsfw" category for sensitive statuses in OStatus serializer (#7048)
Fix #7011
2018-04-22 22:09:03 +02:00
Yamagishi Kazutoshi
648d645c2f Fix randomly fail (similar #7219) (#7225) 2018-04-22 11:41:39 +02:00
Yamagishi Kazutoshi
3f6893c641 Reset locale on registration tests (#7219) 2018-04-21 23:37:07 +02:00
Yamagishi Kazutoshi
d10447c3a8 Use raw status code on have_http_status (#7214) 2018-04-21 21:35:07 +02:00
Yamagishi Kazutoshi
b438224751 Introduce rspec-retry (#7206) 2018-04-20 19:31:30 +02:00
Eugen Rochko
a9c440637c
Improve report layout (#7188)
* Use table for statuses in report

* Display reported account and reporter in the same table

* Split accounts and general report info into two tables again

* Redesign report statuses table, notes, merge notes and action log

* Remove unused translations

* Fix code style issue

* Fix code style issue

* Fix code style issue
2018-04-20 02:28:48 +02:00
Kaito Sinclaire
156b916caf Direct messages column (#4514)
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL

* Changes to match other timelines in 2.0
2018-04-18 13:09:06 +02:00
Akihiko Odaki
a1049e9380 Redirect to account status page for page of status stream entry (#7104)
Commit 519119f657 missed a change for
stream entry page. Instead of duplicating the change, redirect to account
status page. It would also help crawlers (of search engines, for example)
to understand a stream entry URL and its corresponding status URL points
to the same page.
2018-04-16 10:04:31 +02:00
Eugen Rochko
78ed4ab75f
Add bio fields (#6645)
* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121
2018-04-14 12:41:08 +02:00
Renato "Lond" Cerqueira
14d86eb0d0 Allow more than the max pins if account is not local (#7105)
Sidekiq sometimes throws errors for users that have more pinned items
than the allowed by the local instance. It should only validate the
number of pins for local accounts.
2018-04-12 20:36:02 +02:00
Alexander
8e88a18316 update gem, test pam authentication (#7028)
* update gem, test pam authentication

* add description for test parameters

* fix inclusion of optional group
2018-04-11 21:40:38 +02:00
Akihiko Odaki
519119f657 Paginate ancestor statuses in public page (#7102)
This also limits the statuses returned by API, but pagination is not
implemented in Web API yet. I still expect it brings user experience
better than making a user wait to fetch all ancestor statuses and flooding
the column with them.
2018-04-11 12:35:09 +02:00
Paul Woolcock
8f800ad691 Change custom emoji search to ILIKE instead of = (#7099) 2018-04-10 15:46:27 +02:00
Emelia Smith
219a4423d8 Feature: Allow staff to change user emails (#7074)
* Admin: Show unconfirmed email address on account page

* Admin: Allow staff to change user email addresses

* ActionLog: On change_email, log current email address and new unconfirmed email address
2018-04-10 09:16:06 +02:00
Levi Bard
cd0eaa349c Enable updating additional account information from user preferences via rest api (#6789)
* Enable updating additional account information from user preferences via rest api
Resolves #6553

* Pacify rubocop

* Decoerce incoming settings in UserSettingsDecorator

* Create user preferences hash directly from incoming credentials instead of going through ActionController::Parameters

* Clean up user preferences update

* Use ActiveModel::Type::Boolean instead of manually checking stringified number equivalence
2018-04-08 13:43:10 +02:00
Eugen Rochko
07176fed37
Add contact account to landing page ("Administered by") (#6984) 2018-04-04 01:11:26 +02:00
Emelia Smith
2e59751823 Improve require_admin! and require_staff! filters (#7018)
Previously these returns 302 redirects instead of 403s, which meant posting links to admin pages in slack caused them to unfurl, rather than stay as a link. Additionally, require_admin! doesn't appear to be actively used, on require_staff!
2018-04-03 13:07:32 +02:00
Emelia Smith
e85cffb236 Feature: Report improvements (#6967) (#7000)
* Implement Assignment of Reports (#6967)

* Change translation of admin.report.comment.label to "Report Comment" for clarity

As we'll soon add the ability for reports to have comments on them, this clarification makes sense.

* Implement notes for Reports

This enables moderators to leave comments about a report whilst they work on it

* Fix display of report moderation notes

* Allow reports to be reopened / marked as unresolved

* Redirect to reports listing upon resolution of report

* Implement "resolve with note" functionality

* Add inverse relationship for report notes

* Remove additional database querying when loading report notes

* Fix tests for reports

* Fix localisations for report notes / reports
2018-04-02 22:04:14 +02:00
David Underwood
123a343d11 [WIP] Enable custom emoji on account pages and in the sidebar (#6124)
Federate custom emojis with accounts
2018-04-01 23:55:42 +02:00
unarist
a1d0915585 Add a spec for UniqueUsernameValidator (#6927)
Note that this spec has a pending test about dots in the username,
because allowing it has been reverted for now.
2018-03-27 05:22:58 +02:00
Akihiko Odaki
40e5d2303b Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Akihiko Odaki
b2a4ffd3a9 Change columns in notifications nonnullable (#6764) 2018-03-24 12:51:28 +01:00
Akihiko Odaki
54b273bf99 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Rey Tucker
36b5703796 request: in the event of failure, try other IPs (#6761) (#6813)
* request: in the event of failure, try other IPs (#6761)

In the case where a name has multiple A/AAAA records, we should
try subsequent records instead of immediately failing when we have a
failure on the first IP address.

This significantly improves delivery success when there are network
connectivity problems affecting only IPv4 or IPv6.

* fix method call style

* request_spec: adjust test case to use Addrinfo

* request: Request/open: move private addr check to within begin/rescue

* request_spec: add case to test failover, fix exception check

* Double Addrinfo.foreach so that it correctly yields instances
2018-03-20 09:06:08 +01:00
Eugen Rochko
e6520c0270
Fix #6657 - Use target instead of origin in Remove activity (#6664) 2018-03-07 03:54:46 +01:00
Akihiko Odaki
a38dbd9c8a Redirect from Web tag timeline to public tag timeline if not signed in (#6633)
This is also implemented in Pawoo:
ceafdbd1bb
2018-03-05 19:29:36 +01:00
Akihiko Odaki
ef44c62d17 Do not default site_title with site_hostname in InstanceHelper (#6624)
site_title is "Mastodon" by default configuration, and there is no need to
default site_title with site_hostname in InstanceHelper.
2018-03-04 20:29:49 +01:00
Akihiko Odaki
51d760960c Set the default locale in config (#6580)
Previously the default locale was set by Localized concern for controllers,
but it was not enforced for mailers.

config is enforced throughout the application and an appropriate place to
set the default locale.
2018-03-04 09:21:35 +01:00
Eugen Rochko
9110db41c5
Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
Eugen Rochko
44829d8216
Fix missing focalPoint in ActivityPub JSON (#6609) 2018-03-04 07:21:41 +01:00
Eugen Rochko
41a01bec23
Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Akihiko Odaki
2e8a492e88 Raise Mastodon::HostValidationError when host for HTTP request is private (#6410) 2018-02-24 19:16:11 +01:00
Akihiko Odaki
f8f0572ee0 Do not push status to feed if its reblog is already inserted (#6488)
A complemental change for precompute_feed_service_spec.rb also fixes its
random failure which is caused by the Snowlake randomization of the order
of an original status and its reblog.
2018-02-24 05:40:18 +01:00
Eugen Rochko
c71aa468b5
Redesign landing page (again) (#6486)
* Redesign landing page (again)

* Move login form in small version to the right column

* Display closed registrations message

* Add site setting for the hero image

* Fix test

* Increase spacing, maximum width, change call to action section
2018-02-22 01:03:48 +01:00
Eugen Rochko
4bc625166e
Fix bug in relationships API introduced by #6482 (#6527)
It was merge when it needed to be deep_merge. And added some tests
2018-02-21 23:22:12 +01:00
Eugen Rochko
61ed133fea
Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
abcang
7124881273 Improve performance of feed_manager_spec (#6517) 2018-02-20 16:50:12 +01:00