Commit graph

254 commits

Author SHA1 Message Date
kim
94e87610c4
[chore] add back exif-terminator and use only for jpeg,png,webp (#3161)
* add back exif-terminator and use only for jpeg,png,webp

* fix arguments passed to terminateExif()

* pull in latest exif-terminator

* fix test

* update processed img

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-02 12:46:41 +01:00
tobi
8ab2b19a94
[feature] Federate interaction policies + Accepts; enforce policies (#3138)
* [feature] Federate interaction policies + Accepts; enforce policies

* use Acceptable type

* fix index

* remove appendIRIStrs

* add GetAccept federatingdb function

* lock on object IRI
2024-07-26 12:04:28 +02:00
dependabot[bot]
5338825d2b
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.73 to 7.0.74 (#3125)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.73 to 7.0.74.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.73...v7.0.74)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-22 14:12:51 +01:00
dependabot[bot]
16421f7576
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.34 to 2.20.37 (#3106) 2024-07-15 09:15:31 +00:00
dependabot[bot]
b8a291c430
[chore]: Bump github.com/coreos/go-oidc/v3 from 3.10.0 to 3.11.0 (#3104) 2024-07-15 09:14:50 +00:00
kim
cde2fb6244
[feature] support processing of (many) more media types (#3090)
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg

* specify the video codec to use when generating static image from emoji

* update go-storage library (fixes incompatibility after updating go-iotools)

* maintain image aspect ratio when generating a thumbnail for it

* update readme to show go-ffmpreg

* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility

* remove extra debug from error message

* fix up incorrect function signatures

* update PutFile to just use regular file copy, as changes are file is on separate partition

* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe

* update more tests, add more code comments

* add utilities to generate processed emoji / media outputs

* fix remaining tests

* add test for opus media file, add license header to utility cmds

* limit the number of concurrently available ffmpeg / ffprobe instances

* reduce number of instances

* further reduce number of instances

* fix envparsing test with configuration variables

* update docs and configuration with new media-{local,remote}-max-size variables
2024-07-12 09:39:47 +00:00
Daenney
1de41f64f2
[chore] Bump ncruces/go-sqlite3 to 0.17.1 (#3085)
More linkanme fixes.
2024-07-08 22:03:00 +02:00
dependabot[bot]
43c480aec4
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.72 to 7.0.73 (#3083) 2024-07-08 07:59:07 +00:00
dependabot[bot]
5769722c58
[chore]: Bump github.com/microcosm-cc/bluemonday from 1.0.26 to 1.0.27 (#3081) 2024-07-08 07:34:39 +00:00
tobi
1a66ea8998
[chore] upstep activity to v1.7.0-gts (#3074) 2024-07-06 15:57:11 +02:00
Daenney
49009fbd8f
[chore] Update ncruces/go-sqlite3 to 0.17 (#3072)
This fixes some linkname shenanigans previous versions of the library
were using. It's now safe to upgrade to Go 1.23 and beyond once they
become available.
2024-07-05 12:06:03 +02:00
dependabot[bot]
9c44d6aa47
[chore]: Bump github.com/yuin/goldmark from 1.7.3 to 1.7.4 (#3059)
Bumps [github.com/yuin/goldmark](https://github.com/yuin/goldmark) from 1.7.3 to 1.7.4.
- [Release notes](https://github.com/yuin/goldmark/releases)
- [Commits](https://github.com/yuin/goldmark/compare/v1.7.3...v1.7.4)

---
updated-dependencies:
- dependency-name: github.com/yuin/goldmark
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-01 13:41:55 +02:00
dependabot[bot]
03c5d016a7
[chore]: Bump github.com/gorilla/feeds from 1.1.2 to 1.2.0 (#3035) 2024-06-24 09:04:27 +00:00
dependabot[bot]
b9864e83a0
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.71 to 7.0.72 (#3037) 2024-06-24 08:07:15 +00:00
dependabot[bot]
ddef307622
[chore]: Bump github.com/yuin/goldmark from 1.7.2 to 1.7.3 (#3034) 2024-06-24 08:06:44 +00:00
kim
15e0bf6e5a
[chore] update github.com/ncruces/go-sqlite3 -> v0.16.3 (#3029) 2024-06-22 11:59:29 +02:00
kim
9143ac6fb4
updates go-mutexes to no longer rely on unsafe linkname (#3027) 2024-06-21 17:35:32 +01:00
kim
7b1ccbd65a
[feature] add worker task serialization logic (#2989)
* improved server shutdown with more precise shutdown of modules + deferring of ALL of it

* move delivery and workers into separate files

* add worker task model and Serialize() / Deserialize() methods for message types

* start adding message serialize / deserialize tests

* start adding test cases

* update body rewinding to rely on standard library mechanism of r.GetBody()

* remove request rewinding (http.Client{} should already handle this)

* standard library already handles rewinding

* improved code comment

* move the newPOST() function contents to prepare(), fits better with current API

* add Serialize() / Deserialize() implementations for Delivery{} type

* finish writing FromClientAPI sserialize / deserialize tests

* start adding FromFediAPI{} serialize / deserialize test cases

* fix FromFediAPI{} tests

* add tests for delivery message type

* fix repeat code

* missing license header

* use testrig status and accounts for marshal / unmarshaling tests

* add a specific test for checking account RSA keys are preserved
2024-06-20 17:06:20 +00:00
dependabot[bot]
7ab404d643
[chore]: Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 (#3016) 2024-06-17 08:13:40 +00:00
dependabot[bot]
3a01377bcc
[chore]: Bump github.com/miekg/dns from 1.1.59 to 1.1.61 (#3014) 2024-06-17 08:12:25 +00:00
dependabot[bot]
b7dede2a47
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.33 to 2.20.34 (#3017) 2024-06-17 08:11:08 +00:00
dependabot[bot]
76d695d847
[chore]: Bump github.com/yuin/goldmark from 1.7.1 to 1.7.2 (#3018) 2024-06-17 08:10:11 +00:00
Daenney
978b4176f1
[chore] Upgrade wasm-sqlite to v0.16.2 (#2997) 2024-06-12 13:21:34 +01:00
dependabot[bot]
a687f0634c
[chore]: Bump github.com/gorilla/websocket from 1.5.1 to 1.5.2 (#2987)
Bumps [github.com/gorilla/websocket](https://github.com/gorilla/websocket) from 1.5.1 to 1.5.2.
- [Release notes](https://github.com/gorilla/websocket/releases)
- [Commits](https://github.com/gorilla/websocket/compare/v1.5.1...v1.5.2)

---
updated-dependencies:
- dependency-name: github.com/gorilla/websocket
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-10 07:43:38 +00:00
dependabot[bot]
12c406d968
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.32 to 2.20.33 (#2984)
Bumps [github.com/tdewolff/minify/v2](https://github.com/tdewolff/minify) from 2.20.32 to 2.20.33.
- [Release notes](https://github.com/tdewolff/minify/releases)
- [Commits](https://github.com/tdewolff/minify/compare/v2.20.32...v2.20.33)

---
updated-dependencies:
- dependency-name: github.com/tdewolff/minify/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-10 07:41:15 +00:00
dependabot[bot]
83ee766e34
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.70 to 7.0.71 (#2988)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.70 to 7.0.71.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.70...v7.0.71)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-10 07:40:16 +00:00
Daenney
cc4f773b0e
[chore] Update WASM go-sqlite3 to v0.16.1 (#2976)
This includes support for journal mode set to WAL on the BSDs.

Relates to: #1753, #2962
2024-06-07 15:06:43 +02:00
dependabot[bot]
addaba05c0
[chore]: Bump github.com/spf13/viper from 1.18.2 to 1.19.0 (#2954)
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.18.2 to 1.19.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.18.2...v1.19.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/viper
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-03 10:52:54 +01:00
kim
1e7b32490d
[experiment] add alternative wasm sqlite3 implementation available via build-tag (#2863)
This allows for building GoToSocial with [SQLite transpiled to WASM](https://github.com/ncruces/go-sqlite3) and accessed through [Wazero](https://wazero.io/).
2024-05-27 17:46:15 +02:00
dependabot[bot]
d96cca60a1
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.25 to 2.20.32 (#2927) 2024-05-27 09:36:09 +00:00
dependabot[bot]
0a18c0d802
[chore]: Bump github.com/jackc/pgx/v5 from 5.5.5 to 5.6.0 (#2929) 2024-05-27 09:35:41 +00:00
kim
3d3e99ae52
[performance] update storage backend and make use of seek syscall when available (#2924)
* update to use go-storage/ instead of go-store/v2/storage/

* pull in latest version from codeberg

* remove test output 😇

* add code comments

* set the exclusive bit when creating new files in disk config

* bump to actual release version

* bump to v0.1.1 (tis a simple no-logic change)

* update readme

* only use a temporary read seeker when decoding video if required (should only be S3 now)

* use fastcopy library to use memory pooled buffers when calling TempFileSeeker()

* update to use seek call in serveFileRange()
2024-05-22 11:46:24 +02:00
kim
6c0d93c6cb
[chore] dependabot updates (#2922)
* [chore]: Bump github.com/prometheus/client_golang from 1.18.0 to 1.19.1

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.18.0 to 1.19.1.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.18.0...v1.19.1)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [chore]: Bump github.com/KimMachineGun/automemlimit from 0.6.0 to 0.6.1

Bumps [github.com/KimMachineGun/automemlimit](https://github.com/KimMachineGun/automemlimit) from 0.6.0 to 0.6.1.
- [Release notes](https://github.com/KimMachineGun/automemlimit/releases)
- [Commits](https://github.com/KimMachineGun/automemlimit/compare/v0.6.0...v0.6.1)

---
updated-dependencies:
- dependency-name: github.com/KimMachineGun/automemlimit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [chore]: Bump github.com/tdewolff/minify/v2 from 2.20.20 to 2.20.24

Bumps [github.com/tdewolff/minify/v2](https://github.com/tdewolff/minify) from 2.20.20 to 2.20.24.
- [Release notes](https://github.com/tdewolff/minify/releases)
- [Commits](https://github.com/tdewolff/minify/compare/v2.20.20...v2.20.24)

---
updated-dependencies:
- dependency-name: github.com/tdewolff/minify/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* [chore]: Bump github.com/go-swagger/go-swagger

Bumps [github.com/go-swagger/go-swagger](https://github.com/go-swagger/go-swagger) from 0.30.6-0.20240418033037-c46c303aaa02 to 0.31.0.
- [Release notes](https://github.com/go-swagger/go-swagger/releases)
- [Changelog](https://github.com/go-swagger/go-swagger/blob/master/.goreleaser.yml)
- [Commits](https://github.com/go-swagger/go-swagger/commits/v0.31.0)

---
updated-dependencies:
- dependency-name: github.com/go-swagger/go-swagger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* [chore]: Bump github.com/gin-gonic/gin from 1.9.1 to 1.10.0

Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.1 to 1.10.0.
- [Release notes](https://github.com/gin-gonic/gin/releases)
- [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gin-gonic/gin/compare/v1.9.1...v1.10.0)

---
updated-dependencies:
- dependency-name: github.com/gin-gonic/gin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-21 14:17:22 +01:00
dependabot[bot]
16c1832793
[chore]: Bump github.com/gin-contrib/cors from 1.7.1 to 1.7.2 (#2912)
Bumps [github.com/gin-contrib/cors](https://github.com/gin-contrib/cors) from 1.7.1 to 1.7.2.
- [Release notes](https://github.com/gin-contrib/cors/releases)
- [Changelog](https://github.com/gin-contrib/cors/blob/master/.goreleaser.yaml)
- [Commits](https://github.com/gin-contrib/cors/compare/v1.7.1...v1.7.2)

---
updated-dependencies:
- dependency-name: github.com/gin-contrib/cors
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2024-05-13 08:29:54 +00:00
dependabot[bot]
f817f96596
[chore]: Bump github.com/gin-contrib/sessions from 1.0.0 to 1.0.1 (#2916)
Bumps [github.com/gin-contrib/sessions](https://github.com/gin-contrib/sessions) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/gin-contrib/sessions/releases)
- [Changelog](https://github.com/gin-contrib/sessions/blob/master/.goreleaser.yaml)
- [Commits](https://github.com/gin-contrib/sessions/compare/v1.0.0...v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/gin-contrib/sessions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-13 08:07:17 +00:00
kim
c06e6fb656
[performance] update go-structr and go-mutexes with memory usage improvements (#2909)
* update go-structr and go-mutexes with memory usage improvements

* bump to go-structr v0.8.4
2024-05-13 08:05:46 +00:00
dependabot[bot]
a5f28fe0c9
[chore]: Bump github.com/gin-contrib/gzip from 1.0.0 to 1.0.1 (#2899)
Bumps [github.com/gin-contrib/gzip](https://github.com/gin-contrib/gzip) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/gin-contrib/gzip/releases)
- [Changelog](https://github.com/gin-contrib/gzip/blob/master/.goreleaser.yaml)
- [Commits](https://github.com/gin-contrib/gzip/compare/v1.0.0...v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/gin-contrib/gzip
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-06 08:50:47 +00:00
dependabot[bot]
bfc21e4850
[chore]: Bump github.com/tdewolff/minify/v2 from 2.20.19 to 2.20.20 (#2875) 2024-04-29 10:45:17 +00:00
dependabot[bot]
d3bac8bbec
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.69 to 7.0.70 (#2877) 2024-04-29 10:44:11 +00:00
dependabot[bot]
1375a86919
[chore]: Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#2878) 2024-04-29 09:18:16 +00:00
tobi
fd8a724e77
[chore] Bump go swagger (#2871)
* bump go swagger version

* bump swagger version
2024-04-26 11:31:10 +02:00
dependabot[bot]
a57dd15a8e
[chore]: Bump github.com/miekg/dns from 1.1.58 to 1.1.59 (#2861) 2024-04-22 09:11:53 +00:00
dependabot[bot]
0db9e34b69
[chore]: Bump github.com/KimMachineGun/automemlimit from 0.5.0 to 0.6.0 (#2859) 2024-04-22 08:59:30 +00:00
kim
1018cde107
[chore] bump bun library versions (#2837) 2024-04-15 12:01:20 +02:00
dependabot[bot]
c097745c38
[chore]: Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#2818)
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go) from 1.24.0 to 1.25.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.24.0...v1.25.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-11 11:46:18 +02:00
kim
a483bd9e38
[performance] massively improved ActivityPub delivery worker efficiency (#2812)
* add delivery worker type that pulls from queue to httpclient package

* finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller

* hook up queue deletion logic

* support deleting queued http requests by target ID

* don't index APRequest by hostname in the queue

* use gorun

* use the original context's values when wrapping msg type as delivery{}

* actually log in the AP delivery worker ...

* add uncommitted changes

* use errors.AsV2()

* use errorsv2.AsV2()

* finish adding some code comments, add bad host handling to delivery workers

* slightly tweak deliveryworkerpool API, use advanced sender multiplier

* remove PopCtx() method, let others instead rely on Wait()

* shuffle things around to move delivery stuff into transport/ subpkg

* remove dead code

* formatting

* validate request before queueing for delivery

* finish adding code comments, fix up backoff code

* finish adding more code comments

* clamp minimum no. senders to 1

* add start/stop logging to delivery worker, some slight changes

* remove double logging

* use worker ptrs

* expose the embedded log fields in httpclient.Request{}

* ensure request context values are preserved when updating ctx

* add delivery worker tests

* fix linter issues

* ensure delivery worker gets inited in testrig

* fix tests to delivering messages to check worker delivery queue

* update error type to use ptr instead of value receiver

* fix test calling Workers{}.Start() instead of testrig.StartWorkers()

* update docs for advanced-sender-multiplier

* update to the latest activity library version

* add comment about not using httptest.Server{}
2024-04-11 11:45:35 +02:00
dependabot[bot]
15733cddb2
[chore]: Bump github.com/yuin/goldmark from 1.7.0 to 1.7.1 (#2819)
Bumps [github.com/yuin/goldmark](https://github.com/yuin/goldmark) from 1.7.0 to 1.7.1.
- [Release notes](https://github.com/yuin/goldmark/releases)
- [Commits](https://github.com/yuin/goldmark/compare/v1.7.0...v1.7.1)

---
updated-dependencies:
- dependency-name: github.com/yuin/goldmark
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-10 12:04:27 +01:00
kim
adf345f1ec
[chore] bump go structr cache version -> v0.6.0 (#2773)
* update go-structr library -> v0.6.0, add necessary wrapping types + code changes to support these changes

* update readme with go-structr package changes

* improved wrapping of the SliceCache type

* add code comments for the cache wrapper types

* remove test.out 😇

---------

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2024-04-02 12:03:40 +02:00
dependabot[bot]
29031d1e27
[chore]: Bump github.com/gin-contrib/sessions from 0.0.5 to 1.0.0 (#2782) 2024-03-25 11:00:36 +00:00
dependabot[bot]
a24936040c
[chore]: Bump github.com/coreos/go-oidc/v3 from 3.9.0 to 3.10.0 (#2779) 2024-03-25 10:58:34 +00:00