Commit graph

1296 commits

Author SHA1 Message Date
Vyr Cossont
f0a27299f5 Fix type of apimodel.StatusCreateRequest.ScheduledAt in tests 2025-01-26 12:43:11 -08:00
Vyr Cossont
69b0b6ae27 Test that backfills do not cause federation 2025-01-26 12:24:05 -08:00
Vyr Cossont
2971aedd06 Fix check for absence of notification 2025-01-26 11:54:04 -08:00
Vyr Cossont
7c395aeed4 Test that backfilling doesn't timeline or notify 2025-01-26 09:55:23 -08:00
Vyr Cossont
371b351f12 Add new tests for backfilled statuses 2025-01-26 09:21:03 -08:00
Vyr Cossont
227d6f9719 Update error messages & codes 2025-01-26 08:41:25 -08:00
Vyr Cossont
1e9646596b Forbid mentioning others in backfills 2025-01-25 23:24:49 -08:00
Vyr Cossont
a83e4fb7cb Implement backfilling statuses thru scheduled_at 2025-01-25 22:13:25 -08:00
tobi
4c052c85f5
[bugfix] Rename domain perm sub migration to unique date (#3679)
* [bugfix] Rename domain perm sub migration to unique date

* add repeat migration
2025-01-24 18:09:55 +01:00
tobi
71b50353eb
[feature] Process incoming Undo Announce properly (#3676)
* [feature] Process incoming Undo Announce properly

* test undo announce
2025-01-24 17:36:34 +01:00
Vyr Cossont
5b765d734e
[feature] Push notifications (#3587)
* Update push subscription API model to be Mastodon 4.0 compatible

* Add webpush-go dependency

# Conflicts:
#	go.sum

* Single-row table for storing instance's VAPID key pair

* Generate VAPID key pair during startup

* Add VAPID public key to instance info API

* Return VAPID public key when registering an app

* Store Web Push subscriptions in DB

* Add Web Push sender (similar to email sender)

* Add no-op push senders to most processor tests

* Test Web Push notifications from workers

* Delete Web Push subscriptions when account is deleted

* Implement push subscription API

* Linter fixes

* Update Swagger

* Fix enum to int migration

* Fix GetVAPIDKeyPair

* Create web push subscriptions table with indexes

* Log Web Push server error messages

* Send instance URL as Web Push JWT subject

* Accept any 2xx code as a success

* Fix malformed VAPID sub claim

* Use packed notification flags

* Remove unused date columns

* Add notification type for update notifications

Not used yet

* Make GetVAPIDKeyPair idempotent

and remove PutVAPIDKeyPair

* Post-rebase fixes

* go mod tidy

* Special-case 400 errors other than 408/429

Most client errors should remove the subscription.

* Improve titles, trim body to reasonable length

* Disallow cleartext HTTP for Web Push servers

* Fix lint

* Remove redundant index on unique column

Also removes redundant unique and notnull tags on ID column since these are implied by pk

* Make realsender.go more readable

* Use Tobi's style for wrapping errors

* Restore treating all 5xx codes as temporary problems

* Always load target account settings

* Stub `policy` and `standard`

* webpush.Sender: take type converter as ctor param

* Move webpush.MockSender and noopSender into testrig
2025-01-23 16:47:30 -08:00
tobi
9333bbc4d0
[feature] Serve bot accounts over AP as Service instead of Person (#3672)
* pepis

* oopsie doopsie

* bollocks
2025-01-23 18:18:23 +01:00
tobi
b42cb7a802
[feature] Add warning about trusted-proxies to make config easier (#3675)
* [feature] Add warning about `trusted-proxies` to make config easier

* thank you linter, hugs and kisses to you
2025-01-23 15:48:09 +01:00
tobi
1472d92a8d
[feature] Add published property to outgoing AP Actor representations (#3671) 2025-01-23 14:42:31 +00:00
kim
0a99901c65
[performance] reduce InboxForward->Create calls by partially implementing Exists() (#3647)
* alphabetical reordering

* keep a cache of activity IDs we have handled creates for

* reduce number of inbox forwarding create calls by partially implementing Exists()

* increase cache size, since all we're storing is string keys
2025-01-22 13:42:12 +01:00
tobi
37fd7c7a6a
[bugfix] Store LastModified for domain perm subs + send as If-Modified-Since (#3655) 2025-01-20 10:56:00 +01:00
tobi
634d4f408f
[bugfix] Fix existing perm adoption (#3651)
* [bugfix] Fix existing perm adoption

* go fmt

* test, small log fix
2025-01-18 18:55:27 +01:00
kim
e77c7e16b6
[chore] better dns validation (#3644)
* add seperate PunifyValidate() function for properly validating domain names when converting to punycode

* rename function, strip port from domain validation
2025-01-14 14:23:18 +00:00
kim
8cfae010a9
[bugfix] migration to cleanup dropped status edits (#3637)
* add migration to cleanup dropped status edits from previous bug, relinking with statuses

* don't use bun.Ident where not needed

* fix join statement

* fix query to work for both postgres and sqlite

* actually update by the currently set status ID field 🤦
2025-01-09 00:02:22 +00:00
tobi
8daa4dae34
[bugfix] More permissive CSV parsing for perm subs, text parse fix (#3638)
* [bugfix] More permissive CSV parsing for perm subs, text parse fix

* wee

* change the way dry works, slightly

* me oh my, i'm just a little guy

* we're just normal men
2025-01-08 22:38:27 +01:00
tobi
451803b230
[feature] Fetch + create domain permissions from subscriptions nightly (#3635)
* peepeepoopoo

* test domain perm subs

* swagger

* envparsing

* dries your wets

* start on docs

* finish up docs

* copy paste errors

* rename actions package

* rename force -> skipCache

* move obfuscate parse nearer to where err is checked

* make higherPrios a simple slice

* don't use receiver for permsFrom funcs

* add more context to error logs

* defer finished log

* use switch for permType instead of if/else

* thanks linter, love you <3

* validate csv headers before full read

* use bufio scanner
2025-01-08 11:29:40 +01:00
kim
c013892ca2
[chore] replace statuses.updated_at column with statuses.edited_at (#3636)
* update statuses table to replace updated_at column with edited_at

* code comment

* better code comments, fix setting of status + edit + mention + poll database times

* fix log to logf call

* fix status.EditIDs not being carried over in dereferencer.encrichStatus()

* move status.EditID setting into handleStatusEdit()
2025-01-08 11:29:23 +01:00
tobi
e9bb7ddd3a
[feature] Create/update/remove domain permission subscriptions (#3623)
* [feature] Create/update/remove domain permission subscriptions

* lint

* envparsing

* remove errant fmt.Println

* create drafts, subs, exclude, from snapshot models

* name etag column correctly

* remove count column

* lint
2025-01-05 13:20:33 +01:00
Daenney
77f1e79532
[chore] Update robots.txt with more AI bots (#3634) 2025-01-03 11:46:59 +00:00
kim
29da5b35a5
[bugfix] more little edit fixes (#3633)
* fix slice size calculation

* stop attached status edit slice getting cached
2024-12-31 10:44:07 +01:00
Daenney
6b727404be
[bugfix] Advertise audio/mpeg as supported (#3632)
This will be either an mp1, mp2 or mp3 file. In practice it'll probably
be mp3, but this handles mp1 too for good measure. We don't advertise
audio/mp1 as a supported media type since best I can tell that was never
a MIME type that's been used.

This also changes the returned MIME-type for mp2 and mp3 to audio/mpeg,
to match what's expected and supported by most things nowadays.

Fixes: #3531
2024-12-28 11:02:12 +01:00
kim
0784aa3218
[bugfix] small editing tweaks (#3631)
* ensure edited_at isn't set on boost wrapper statuses

* improve handling of remote status updated_at to fix previous cases

* formatting

* add remote status published / updated field validation checks, handle appropriately in handleStatusEdit()

* specifically allowed updated to be equal to published

* only check creation date change when an existing status
2024-12-24 21:16:49 +00:00
kim
fe8d5f2307
[feature] add support for clients editing statuses and fetching status revision history (#3628)
* start adding client support for making status edits and viewing history

* modify 'freshest' freshness window to be 5s, add typeutils test for status -> api edits

* only populate the status edits when specifically requested

* start adding some simple processor status edit tests

* add test editing status but adding a poll

* test edits appropriately adding poll expiry handlers

* finish adding status edit tests

* store both new and old revision emojis in status

* add code comment

* ensure the requester's account is populated before status edits

* add code comments for status edit tests

* update status edit form swagger comments

* remove unused function

* fix status source test

* add more code comments, move media description check back to media process in status create

* fix tests, add necessary form struct tag
2024-12-23 17:54:44 +00:00
Daenney
1aa7f70660
[chore] Stub /api/v1/announcements implementation (#3630)
* [chore] Stub /api/v1/announcements implementation

This implements the /api/v1/announcements endpoint by simply returning
an empty array. This indicates there are no instance announcements.

Some clients retrieve this endpoint and get surprised by a 404. It tends
to be harmless, but results in some unnecessary logging when trying to
debug other things.

* [fix] Forgot some swagger

* [fix] Fix swagger some more
2024-12-23 16:43:50 +01:00
CDN
f78002f915
[bugfix] Load instance-wide custom css in page stylesheets template (#3601)
* [bugfix] Load instance-wide custom css in page stylesheets template

* [chore] remove redunt import
2024-12-18 10:47:17 +01:00
tobi
acdd495c70
[chore] Log during potentially long migrations (#3613)
* [chore] Log during potentially long migrations

* remove unneeded log lines
2024-12-09 15:53:56 +01:00
tobi
9477fd7eba
[feature] Allow partial-word hashtags using non-breaking spaces (#3606)
* [feature] Allow partial-word hashtags using non-breaking spaces

* update docs
2024-12-08 16:03:00 +01:00
Patrycja
642f5230e6
[chore] stub /api/v1/accounts/{id}/featured_tags endpoint (#3598)
* [chore] stub /api/v1/accounts/{id}/featured_tags endpoint

* fix swagger parsing issue

---------

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-12-08 13:47:07 +01:00
kim
23fc70f4e6
[feature] add support for receiving federated status edits (#3597)
* add support for extracting Updated field from Statusable implementers

* add support for status edits in the database, and update status dereferencer to handle them

* remove unused AdditionalInfo{}.CreatedAt

* remove unused AdditionalEmojiInfo{}.CreatedAt

* update new mention creation to use status.UpdatedAt

* remove mention.UpdatedAt, fixes related to NewULIDFromTime() change

* add migration to remove Mention{}.UpdatedAt field

* add migration to add the StatusEdit{} table

* start adding tests, add delete function for status edits

* add more of status edit migrations, fill in more of the necessary edit delete functionality

* remove unused function

* allow generating gotosocial compatible ulid via CLI with `go run ./cmd/gen-ulid`

* add StatusEdit{} test models

* fix new statusedits sql

* use model instead of table name

* actually remove the Mention.UpdatedAt field...

* fix tests now new models are added, add more status edit DB tests

* fix panic wording

* add test for deleting status edits

* don't automatically set `updated_at` field on updated statuses

* flesh out more of the dereferencer status edit tests, ensure updated at field set on outgoing AS statuses

* remove media_attachments.updated_at column

* fix up more tests, further complete the dereferencer status edit tests

* update more status serialization tests not expecting 'updated' AS property

* gah!! json serialization tests!!

* undo some gtscontext wrapping changes

* more serialization test fixing 🥲

* more test fixing, ensure the edit.status_id field is actually set 🤦

* fix status edit test

* grrr linter

* add edited_at field to apimodel status

* remove the choice of paging on the timeline public filtered test (otherwise it needs updating every time you add statuses ...)

* ensure that status.updated_at always fits chronologically

* fix more serialization tests ...

* add more code comments

* fix envparsing

* update swagger file

* properly handle media description changes during status edits

* slight formatting tweak

* code comment
2024-12-05 13:35:07 +00:00
kim
3e18d97a6e
[feature] unending polls (#3592)
* adds support for unending polls to be created locally

* remove unused argument
2024-12-04 10:35:48 +01:00
CDN
732b03a742
[bugfix] Load instance-wide custom css unconditionally in thread details page (#3595) 2024-12-03 15:21:47 +01:00
Victor Dyotte
9609c4550d
[feature] Add global instance CSS customization setting (#3352)
Allow instance admins to add custom CSS that will affect
every page of their instance.

This is done with a new CustomCSS instance setting that
works pretty much exactly like the Users CustomCSS property.
This custom CSS is then requested for every page load.
User styles/themes take precedence over this CSS.

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-12-02 12:24:48 +01:00
kim
c9d36f7e45
[performance] use new instance of bun.DB *after* migrations to reduce number of in-memory model schema (#3578)
* use new instance of bun.DB *after* migrations to reduce number of model schema in-memory

* update sqlite address comment
2024-11-29 15:03:10 +00:00
kim
d9f67efae5
send out poll votes as separate create activities given that no other AP servers support multiple objects in a single activity (#3582) 2024-11-28 15:37:37 +00:00
kim
312cb8b9c7
[chore] rename New___(string) int signature functions to Parse___(string) int (#3580)
* rename New___(string) int {} signature functions to Parse___(string) int {}

* remove test output
2024-11-28 12:54:22 +01:00
tobi
65917f5bb9
[bugfix] Log + ignore unknown notification types (#3577)
* [bugfix] Log + ignore unknown notification types

* pass context to ParseNotificationTypes
2024-11-27 17:22:45 +00:00
Vyr Cossont
6a8af42647
[bugfix] Allow unsetting filter expiration dates (#3560)
* Regression tests for #3497 (v1 and v2)
* use Nullable type for v2 form.expires_in

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-11-26 08:23:00 -08:00
kim
a444adee97
[bugfix] notification types missing from link header (#3571)
* ensure notification types get included in link header query for notifications

* fix type query keys
2024-11-25 15:33:21 +00:00
tobi
c454b1b488
[chore] Bump tooling versions, bump go -> v1.23.0 (#3258)
* [chore] Bump tooling versions, bump go -> v1.23.0

* undo silly change

* sign

* bump go version in go.mod

* allow overflow in imaging

* goreleaser deprecation notices

* bump versions

* undo accidental rebase change

* update container versions to just use latest major version

* update swagger to our release with go1.23 fix

* update goreleaser to use our vendored swagger version

---------

Co-authored-by: kim <grufwub@gmail.com>
2024-11-25 16:15:33 +01:00
kim
cac9d65029
[performance] convert enum strings to ints (#3558)
* convert statuses.visibility and notifications.notification_type columns from type string -> int for performance / space savings

* fix test trying to compare string to int

* fix instance count query using string literal instead of gtsmodel const type

* ensure a default value is always set

* also migrate the account settings and sin bin status tables

* initialize maps outside loops and place into singular enum mapping creation func

* use int16 for enum types

* update sinbinstatus creation to be from a snapshot at initial creation

* add snapshot of poll type at creation time
2024-11-25 14:48:59 +01:00
tobi
301543616b
[feature] Add domain permission drafts and excludes (#3547)
* [feature] Add domain permission drafts and excludes

* fix typescript complaining

* lint

* make filenames more consistent

* test own domain excluded
2024-11-21 13:09:58 +00:00
tobi
c2029df9bc
[feature] Allow emoji shortcode to be 1-character length (#3556)
* [feature] Allow emoji shortcode to be 1-character length

* testerino fixeroni

* spaghet
2024-11-21 12:13:55 +01:00
Jannis
9ace025da1
[bugfix] post counters should not include direct messages (#3554)
* [bugfix] post counters should not include direct messages #3504

The fix is relativly simple, it just adds a line to the relevant
function which excludes all private posts.

* Formating fix

* mb
2024-11-21 11:06:06 +01:00
kim
e3c2b790fd
[performance] minimise log field allocations (#3529)
* when appending log field only do so by minimal amount

* move slice utils to separate package to fix import cycle, add GrowJust() and AppendJust() functions

* fix GrowJust() not returning slice of same length

* improved xslices tests

* make AppendJust() test check for slice contents, fix AppendJust() final copying behaviour

* add a +1 with field growth to try minimise allocation for log 'msg' field
2024-11-11 15:45:19 +00:00
kim
b84637801a
[chore] update go ffmpreg to v0.6.0 (#3515)
* pull in go-ffmpreg v0.6.0

* add code comment

* grrr linter

* set empty module name when calling ffmpeg / ffprobe
2024-11-06 14:38:13 +01:00