#13 Systemd services

Closed
lecris wants to merge 0 commits from lecris:systemd into systemd
lecris commented 3 months ago

Here's a start to provide some basic systemd service files. There are a few issues still needing to be addressed, but let's open it up for discussion to see how to address them:

Discussion:

  • Should either line.service or line-chrome.service be made into socket? As I understand when we make a socket, it can be automatically started and stopped when required to listen. For the line-chrome that might be unproductive because it would require to re-login unless the status can be saved in between sessions, but also it would not be able to recieve updates.
  • Maybe these should be templetized and then generated by setup.py. Are there any standard template formats we can use? I like the ones in github.com/smallstep/certificate
  • Depending on how you want to integrate multiple chrome clients, the line-chrome.service might have to be aliased, i.e. line-chrome@.service, and somehow allowed to be controlled by line.service user via sudoer.

Slight rant:

Currently I don't think there is a stadndard synapse/bridge service infrastructure, and ideally the bridges and plugins should be designed so they can be invoked and editted as something like matrix-bridges@.service, but that's a discussion for the core matrix community later on, e.g. if they can provide a matrix-server.target that we can later link to add dependencies and so on.

Here's a start to provide some basic systemd service files. There are a few issues still needing to be addressed, but let's open it up for discussion to see how to address them: - [ ] `User`: Currently there is no standard matrix/bridge users for synapse or dendrite servers. Some concerns are if there is a common user for both the server and bridges, is there a risk of interfering with each other or any security concerns. - [x] `line-chrome.target` should wait for when `line-chrome.service` is starting to listen. Currently `line.service` fails because `line-chrome.target` is activated too fast. Not sure how to achieve that yet. - [x] Check if erroring and `systemctl start/stop` interact as they're supposed to: - [x] `line-chrome.service` can be start independently - [x] `start line.service` triggers `start line-chrome.service` - [x] `restart line.service` does not restart `line-chrome.service`: Fails - [x] `stop line-chrome.service` stops `line.service` - [x] `restart line-chrome.service` restarts `line.service` - [x] `stop line.service` stops `line-chrome.service` - [ ] ~~Link to the `matrix homeserver` service with similar logic as above~~ - [ ] Confirm that it works on other OS (just need to check executable paths): - [x] RHEL - [ ] Debian - [ ] Add logging? ## Discussion: - Should either `line.service` or `line-chrome.service` be made into socket? As I understand when we make a socket, it can be automatically started and stopped when required to listen. For the `line-chrome` that might be unproductive because it would require to re-login unless the status can be saved in between sessions, but also it would not be able to recieve updates. - Maybe these should be templetized and then generated by `setup.py`. Are there any standard template formats we can use? I like the ones in `github.com/smallstep/certificate` - Depending on how you want to integrate multiple chrome clients, the `line-chrome.service` might have to be aliased, i.e. `line-chrome@.service`, and somehow allowed to be controlled by `line.service` user via `sudoer`. ## Slight rant: Currently I don't think there is a stadndard synapse/bridge service infrastructure, and ideally the bridges and plugins should be designed so they can be invoked and editted as something like `matrix-bridges@.service`, but that's a discussion for the core matrix community later on, e.g. if they can provide a `matrix-server.target` that we can later link to add dependencies and so on.
lecris added 1 commit 3 months ago
fair commented 3 months ago
Poster
Owner

Thanks for this!
Some comments:

  • Regarding User: it's probably enough to have a commented-out User (as you already did) to not break default installs, while still suggesting the best-practice of running with a system user.
  • To handle its dependency on line-chrome.service, maybe line.service can just use Restart=always to make it keep trying to connect to the socket until it succeeds. Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure.
  • Any dependencies on a homeserver unit are IMO out-of-scope for this, given that there's no way to predict which (or how many) homeservers someone has installed, and like you said, there's no standard target/service unit naming scheme for HSs. Plus, this bridge should ideally poll for a HS connection instead of fast-failing, and shouldn't fail or need to restart if the HS crashes (like most bridges), so it may not need a hard dependency on a homeserver at all.
  • Regarding using a socket unit, I was planning on having the node process manage multiple sockets itself, since it looks like most of the connection-management code can already handle multiple connections. Socket units do sound like a good idea though, so I'll think about it more before deciding anything.
Thanks for this! Some comments: * Regarding `User`: it's probably enough to have a commented-out `User` (as you already did) to not break default installs, while still suggesting the best-practice of running with a system user. * To handle its dependency on `line-chrome.service`, maybe `line.service` can just use `Restart=always` to make it keep trying to connect to the socket until it succeeds. Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure. * Any dependencies on a homeserver unit are IMO out-of-scope for this, given that there's no way to predict which (or how many) homeservers someone has installed, and like you said, there's no standard target/service unit naming scheme for HSs. Plus, this bridge should ideally poll for a HS connection instead of fast-failing, and shouldn't fail or need to restart if the HS crashes (like most bridges), so it may not need a hard dependency on a homeserver at all. * Regarding using a socket unit, I was planning on having the node process manage multiple sockets itself, since it looks like most of the connection-management code can already handle multiple connections. Socket units do sound like a good idea though, so I'll think about it more before deciding anything.
lecris commented 3 months ago
Poster

Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure.

That would indeed be preferred. A n time retry with a few seconds time-out should do. If possible I would prefer to avoid Restart=always because it might hide some underlying errors down the line.

Any dependencies on a homeserver unit are IMO out-of-scope for this

Agree, would be nice to have something standardized from the matrix dev team, but until then it is not necessary. Also it should be able to be run with external server in the future like via Dimension or the other pluggable method, so I steadily come to think that it is uneccessary, at least for bridges and such.

Regarding using a socket unit

If node could manage that it would be ideal. One problem with using systemd socket is that it has to be fast to start and persistent. Probably the only place it makes sense to have is on the line.service, but the effect there would be minimal?

Also how about the locations? It would be easy to create these from a template when setting up the database.

> Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure. That would indeed be preferred. A n time retry with a few seconds time-out should do. If possible I would prefer to avoid `Restart=always` because it might hide some underlying errors down the line. > Any dependencies on a homeserver unit are IMO out-of-scope for this Agree, would be nice to have something standardized from the matrix dev team, but until then it is not necessary. Also it should be able to be run with external server in the future like via `Dimension` or the other pluggable method, so I steadily come to think that it is uneccessary, at least for bridges and such. > Regarding using a socket unit If node could manage that it would be ideal. One problem with using systemd socket is that it has to be fast to start and persistent. Probably the only place it makes sense to have is on the `line.service`, but the effect there would be minimal? Also how about the locations? It would be easy to create these from a template when setting up the database.
lecris commented 2 months ago
Poster

Should check github.com/matrix-org/synapse#9803 for seucrity hardening tips and tools like systemd-analyze security from this blog

Should check [github.com/matrix-org/synapse#9803](https://github.com/matrix-org/synapse/pull/9803) for seucrity hardening tips and tools like `systemd-analyze security` from [this blog](https://www.ctrl.blog/entry/systemd-service-hardening.html)
lecris added 1 commit 2 months ago
lecris commented 2 months ago
Poster

Update: with this change, line-chrome is active only when the initialization is finished by node.

There might be a depandancy on systemd is there any check to see if it still works without it? Maybe the sd.notify will just send commands to the void so it might still work.

Update: with this change, `line-chrome` is active only when the initialization is finished by node. There might be a depandancy on `systemd` is there any check to see if it still works without it? Maybe the `sd.notify` will just send commands to the void so it might still work.
lecris added 1 commit 2 months ago
lecris force-pushed systemd from 9c09285045 to ea56f82f51 2 months ago
lecris added 1 commit 2 months ago
lecris changed title from WIP: Systemd services to Systemd services 2 months ago
lecris added 1 commit 2 months ago
lecris commented 2 months ago
Poster

The only other options to set are setting Restart options, which I don't know if it should be added atm or later, and hardening the service files, which is quite a slog to do. Will make a separate file in this branch afterwards.

Should be ok to merge like this, but check the yarn settings on your end as well.

The only other options to set are setting `Restart` options, which I don't know if it should be added atm or later, and hardening the service files, which is quite a slog to do. Will make a separate file in this branch afterwards. Should be ok to merge like this, but check the `yarn` settings on your end as well.
lecris added 5 commits 1 month ago
lecris force-pushed systemd from 968832545e to 9d4f2dafac 1 month ago
lecris force-pushed systemd from 9d4f2dafac to 3c0dd15616 1 month ago
fair changed target branch from master to systemd 3 weeks ago
fair commented 3 weeks ago
Poster
Owner

Merged to new systemd branch.

Merged to new `systemd` branch.
fair closed this pull request 3 weeks ago
This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
No reviewers
No Label
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This pull request currently doesn't have any dependencies.

Loading…
There is no content yet.