Commit graph

79 commits

Author SHA1 Message Date
Gergely Nagy
bf2036a32e
Try to handle errors a bit more gracefully
Some checks failed
build / binary (push) Has been cancelled
build / binary-static (x86_64-linux) (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
lint / linting (push) Has been cancelled
In many cases where iocaine was using `unwrap()`, handle it gracefully,
and return a `Result` instead.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-27 08:23:33 +01:00
Gergely Nagy
d291d508cb
Merge a few functions
Some checks failed
build / binary (push) Has been cancelled
build / binary-static (x86_64-linux) (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
lint / linting (push) Has been cancelled
There's no need to have `poison()`, `poison_root()`, and
`poison_path()`: it is enough to have one, and mark path as
`Option<Path<String>>`, and supply a default.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-24 22:46:48 +01:00
Gergely Nagy
d8cc61e636
nix: Move the NixOS module elsewhere
Some checks failed
build / binary (push) Has been cancelled
build / binary-static (x86_64-linux) (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
lint / linting (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-18 11:29:31 +01:00
Gergely Nagy
e2218c6209
Move the docs to a separate repo
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-18 10:50:47 +01:00
Gergely Nagy
9b460216dc
Fix an old reference to algernon/iocaine
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-18 10:26:56 +01:00
Gergely Nagy
b294af5f8e
Move the source to the iocaine organization
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-18 09:15:14 +01:00
Gergely Nagy
2f0a1592d9
docs: Update the Dashboard screenshot
Some checks failed
documentation / documentation (push) Has been cancelled
lint / linting (push) Has been cancelled
documentation / notification (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-15 14:31:20 +01:00
Gergely Nagy
6518013b9c
data: Grafana dashboard update
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-15 14:29:00 +01:00
Gergely Nagy
efa3a203ba
metrics: Provide an iocaine_garbage_served metric, too
Some checks failed
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
build / binary (push) Has been cancelled
build / binary-static (x86_64-linux) (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-15 14:14:55 +01:00
Gergely Nagy
87c1dfa04b
docs: Correct a few copy & paste mistakes
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-15 14:14:25 +01:00
Gergely Nagy
9b41783d58
docs: Document [sources].markov better
Attempt to explain how iocaine uses the sources listed therein, their
format, and give a few examples too.

Fixes #12.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-15 08:28:29 +01:00
Gergely Nagy
98f3ce1f96
Update the Grafana dashboard
Some checks failed
lint / linting (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-12 10:59:44 +01:00
Gergely Nagy
1c24174e8f
Add SPDX header to wurstsalat_generator_pro/substrings.rs
Some checks failed
build / binary (push) Has been cancelled
build / binary-static (x86_64-linux) (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
documentation / documentation (push) Has been cancelled
lint / linting (push) Has been cancelled
documentation / notification (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-10 18:31:36 +01:00
Gergely Nagy
d2f23ec9c8 Merge pull request 'Do less allocating and copying when generating text' (#9) from iadd/iocaine:optimize-markov-generation into main 2025-02-10 17:19:17 +00:00
iadd
284af56e68 Do less allocating and copying when generating text
Before, on a low-capacity system (such as a an inexpensive cloud host),
doing Markov-chain text generation was _extraordinarily_ slow, taking
half a second or more to produce a page, and if multiple requests came
in simultaneously they could easily swamp the capacity of such a system.

Most of the time was spent in the Words iterator, which did a bunch of
cloning of Strings in what the hot path.

This changes the Markov generator's internal representation - now, instead
of storing Strings, it stores index-pairs into a single shared String,
normalized so that all references to particular words are collapsed into
a single pair.  This also means that the hash map is working with
fixed-size values, which can't hurt.

In addition, it does only one hash-map lookup per generated word in the
happy-path of not reaching the end of the chain.

The upshot of all this is that where it was taking a half-second or more
to generate a page, it now takes about 0.001 seconds.

On the downside, the initialization of WurstsalatGeneratorPro has become
rather less flexible.  Before, you created one and then taught it various
strings, or gave it a list of paths to read and teach itself from.  Now,
the _only_ way to create one is directly with a list of paths.  Changing
this is possible, but it means `Substr` would have to learn to distinguish
which source data it came from, which would mean a likely 50% increase in
its size.  It didn't seem worth it to preserve that capability, which
wasn't even being used.
2025-02-10 08:18:54 -08:00
Gergely Nagy
d2bdbf750a
ci: Update to use cellar
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-10 14:20:37 +01:00
Gergely Nagy
5cab905ea2 flake.lock: Update
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (x86_64-linux) (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9c6b49aeac36e2ed73a8c472f1546f6d9cf1addc' (2025-01-14)
  → 'github:NixOS/nixpkgs/f5a32fa27df91dfc4b762671a0e0a859a8a0058f' (2025-02-06)
• Updated input 'pre-commit-hooks':
    'github:cachix/pre-commit-hooks.nix/a5a961387e75ae44cc20f0a57ae463da5e959656' (2025-01-03)
  → 'github:cachix/pre-commit-hooks.nix/9364dc02281ce2d37a1f55b6e51f7c0f65a75f17' (2025-01-21)
• Updated input 'treefmt-nix':
    'github:numtide/treefmt-nix/13c913f5deb3a5c08bb810efd89dc8cb24dd968b' (2025-01-06)
  → 'github:numtide/treefmt-nix/4f09b473c936d41582dd744e19f34ec27592c5fd' (2025-02-07)
2025-02-09 11:17:25 +01:00
Gergely Nagy
75e64d51d8
data: Update the grafana dashboard
Some checks are pending
lint / linting (push) Waiting to run
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-09 09:48:28 +01:00
Gergely Nagy
7c6ac855d8
Lift out the metrics into its own module
Some checks failed
build / binary (push) Has been cancelled
build / binary-static (x86_64-linux) (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
lint / linting (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-07 15:24:22 +01:00
Gergely Nagy
6fa438ebcb
Make the metrics always available
Previously, metrics were behind a feature flag (which was enabled by
default), now it's always available, but still disabled by default.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-07 15:07:40 +01:00
Gergely Nagy
9dfe654ef0
data: Update the Grafana dashboard
Some checks failed
build / binary (push) Waiting to run
build / binary-static (x86_64-linux) (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
lint / linting (push) Waiting to run
documentation / documentation (push) Has been cancelled
documentation / notification (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-06 22:07:17 +01:00
Gergely Nagy
4cc634e07e
metrics: No need to set instance
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-06 22:01:54 +01:00
Gergely Nagy
074a033a02
docs: Link to the monitoring HOWTO from config
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-06 21:52:05 +01:00
Gergely Nagy
4b91d25a88
metrics: Add a process_start_time_seconds gauge
The new metric is there to help gauge iocaine's uptime, without relying
on an external collector like `systemd_exporter`.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-06 21:50:53 +01:00
Gergely Nagy
7497c955fc
app: Do not ignore app startup errors
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (x86_64-linux) (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-06 17:11:46 +01:00
Gergely Nagy
2bdfa92d3e
Implement per-host templates
Fixes #7.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-06 17:04:11 +01:00
Gergely Nagy
5be1a9169e
docs: Add a HOWTO about monitoring
Some checks failed
build / binary (push) Has been cancelled
build / binary-static (x86_64-linux) (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
documentation / documentation (push) Has been cancelled
lint / linting (push) Has been cancelled
documentation / notification (push) Has been cancelled
Also added the latest dashboard export, as an example.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-05 11:26:49 +01:00
Gergely Nagy
0b1ee4fe42
Metrics reimagined
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-05 09:50:07 +01:00
Gergely Nagy
5b43f0e585
docs: Minor formatting fix
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (x86_64-linux) (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-05 03:33:11 +01:00
Gergely Nagy
3126882a35
Implement Prometheus-compatible, optional metrics
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-05 02:36:13 +01:00
Gergely Nagy
099cefb927
ci: Prepare for cross-building binaries
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (x86_64-linux) (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
Include the system name in the cross-built binary file, and put it under
`iocaine-binaries/latest`, rather than `iocaine/${{ version }}`, to
match how the container image is tagged. Releases will have their own
version.

Documentation updated to point to the new place.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-04 08:47:57 +01:00
Gergely Nagy
0b76a1633e
No need for a Mutex
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
lint / linting (push) Waiting to run
There's no need for a `Mutex` in `StatefulIocaine`, so drop it - keep
only `Arc`.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-04 01:43:04 +01:00
Gergely Nagy
a0410a528d
Slightly reduce the amount of cloning
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
lint / linting (push) Waiting to run
Rather than cloning `Iocaine` (along with the markov chain & everything)
all over the place, declare it shared state behind an `Arc<Mutex>`. This
should reduce the amount of cloning done noticably.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-02-03 23:37:03 +01:00
Gergely Nagy
ff9c31d326
ci: Run the build workflow when tests change
Some checks failed
build / binary (push) Has been cancelled
build / binary-static (push) Has been cancelled
build / container (push) Has been cancelled
build / clippy (push) Has been cancelled
lint / linting (push) Has been cancelled
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-30 21:30:39 +01:00
Gergely Nagy
9cb1afb5f0
tests: Add a new test
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-30 21:29:55 +01:00
Gergely Nagy
aa4c387605
Add a number of tests
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
lint / linting (push) Waiting to run
These are quite... simple, and cover little, but it's more than nothing!

While there, enable tests in the Nix package, so they're run in CI too.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-30 09:18:35 +01:00
Gergely Nagy
a039cf2310
Update rand & rand_chacha to 0.9.0
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-30 07:47:39 +01:00
Gergely Nagy
22166e0e7c
docs: Further updates & fixes to the nginx docs
Some checks failed
lint / linting (push) Waiting to run
documentation / documentation (push) Has been cancelled
documentation / notification (push) Has been cancelled
Thanks to @redshiftltd for test driving the nginx setup, and providing
very valuable feedback & the updated configurations.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 19:24:33 +01:00
Gergely Nagy
2050158c5a
docs: Mention the auto-built binaries
Some checks are pending
build / binary (push) Waiting to run
build / binary-static (push) Waiting to run
build / container (push) Waiting to run
build / clippy (push) Waiting to run
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 08:49:10 +01:00
Gergely Nagy
b73fccda2b
ci: Fix the binary upload, hopefully
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 08:37:33 +01:00
Gergely Nagy
0babaecd80
ci: typo fix
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 08:29:43 +01:00
Gergely Nagy
43af527368
nix: Add zstd to the devshell
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 08:23:40 +01:00
Gergely Nagy
ca1a3eb22a
ci: build & publish a static binary too
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 08:16:36 +01:00
Gergely Nagy
ddd948171c
nix: Provide a static build too
While there, make the container use the statically linked package.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 07:59:04 +01:00
Gergely Nagy
ea9972dbfc
docs: Fix the nginx rewrite
With the previous nginx configuration, all requests that matched
`$badagent` were rewrote to `/ai`, without the original URL anywhere.
This resulted in `iocaine` only seeing `/ai`, and thus, every page ended
up being rendered the same.

The rewrite should take the original request URI into account, which is
what this patch does.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 07:44:34 +01:00
Gergely Nagy
f486b99984
Improved documentation
Some checks are pending
build / build (push) Waiting to run
build / clippy (push) Waiting to run
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
Added a few words about compiling Iocaine, added a docker example with a
TOML configuration file (vs the existing environment variable-based
example), and collected a mistake describing the shape of
`IOCAINE__SOURCES__MARKOV`.

These together should largely address #2.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 01:14:58 +01:00
Gergely Nagy
ae8b06a4d5
Make templating actually useful
This rebuilds the templating so that the *content* is no longer
pre-generated, only the parameters. It is up to the template (and some
newly implemented helper functions) to construct the output from those.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-29 00:20:21 +01:00
Gergely Nagy
1f5d4c8a5b
ci/build: Trigger the build for templates/** too
Some checks are pending
build / build (push) Waiting to run
build / clippy (push) Waiting to run
documentation / documentation (push) Waiting to run
documentation / notification (push) Blocked by required conditions
lint / linting (push) Waiting to run
Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-28 09:02:21 +01:00
Gergely Nagy
ed1f5d0a3a
templates: Fix the link template
This was originally an embedded string in Rust, which needed to escape
the quotation mark. The escaping is no longer needed in raw HBS.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-28 09:01:45 +01:00
Gergely Nagy
5d60b84300
Make the generated HTML templatable
Use Handlebars to generate the HTML output, to make it easier to change.

Signed-off-by: Gergely Nagy <me@gergo.csillger.hu>
2025-01-28 08:50:22 +01:00