fair/matrix-puppeteer-line
fair/matrix-puppeteer-line
3
0
Fork 3
Code Issues 26 Pull Requests 2 Projects Releases Wiki Activity

Systemd services #13

Closed
lecris wants to merge 4 commits from systemd into systemd
pull from: systemd
merge into: fair:systemd
Conversation 6 Commits 4 Files Changed 6 +69
lecris commented 2021-05-18 21:39:18 -04:00
Contributor
Copy Link

Here's a start to provide some basic systemd service files. There are a few issues still needing to be addressed, but let's open it up for discussion to see how to address them:

  • User: Currently there is no standard matrix/bridge users for synapse or dendrite servers. Some concerns are if there is a common user for both the server and bridges, is there a risk of interfering with each other or any security concerns.
  • line-chrome.target should wait for when line-chrome.service is starting to listen. Currently line.service fails because line-chrome.target is activated too fast. Not sure how to achieve that yet.
  • Check if erroring and systemctl start/stop interact as they're supposed to:
    • line-chrome.service can be start independently
    • start line.service triggers start line-chrome.service
    • restart line.service does not restart line-chrome.service: Fails
    • stop line-chrome.service stops line.service
    • restart line-chrome.service restarts line.service
    • stop line.service stops line-chrome.service
  • Link to the matrix homeserver service with similar logic as above
  • Confirm that it works on other OS (just need to check executable paths):
    • RHEL
    • Debian
  • Add logging?

Discussion:

  • Should either line.service or line-chrome.service be made into socket? As I understand when we make a socket, it can be automatically started and stopped when required to listen. For the line-chrome that might be unproductive because it would require to re-login unless the status can be saved in between sessions, but also it would not be able to recieve updates.
  • Maybe these should be templetized and then generated by setup.py. Are there any standard template formats we can use? I like the ones in github.com/smallstep/certificate
  • Depending on how you want to integrate multiple chrome clients, the line-chrome.service might have to be aliased, i.e. line-chrome@.service, and somehow allowed to be controlled by line.service user via sudoer.

Slight rant:

Currently I don't think there is a stadndard synapse/bridge service infrastructure, and ideally the bridges and plugins should be designed so they can be invoked and editted as something like matrix-bridges@.service, but that's a discussion for the core matrix community later on, e.g. if they can provide a matrix-server.target that we can later link to add dependencies and so on.

Here's a start to provide some basic systemd service files. There are a few issues still needing to be addressed, but let's open it up for discussion to see how to address them: - [ ] `User`: Currently there is no standard matrix/bridge users for synapse or dendrite servers. Some concerns are if there is a common user for both the server and bridges, is there a risk of interfering with each other or any security concerns. - [x] `line-chrome.target` should wait for when `line-chrome.service` is starting to listen. Currently `line.service` fails because `line-chrome.target` is activated too fast. Not sure how to achieve that yet. - [x] Check if erroring and `systemctl start/stop` interact as they're supposed to: - [x] `line-chrome.service` can be start independently - [x] `start line.service` triggers `start line-chrome.service` - [x] `restart line.service` does not restart `line-chrome.service`: Fails - [x] `stop line-chrome.service` stops `line.service` - [x] `restart line-chrome.service` restarts `line.service` - [x] `stop line.service` stops `line-chrome.service` - [ ] ~~Link to the `matrix homeserver` service with similar logic as above~~ - [ ] Confirm that it works on other OS (just need to check executable paths): - [x] RHEL - [ ] Debian - [ ] Add logging? ## Discussion: - Should either `line.service` or `line-chrome.service` be made into socket? As I understand when we make a socket, it can be automatically started and stopped when required to listen. For the `line-chrome` that might be unproductive because it would require to re-login unless the status can be saved in between sessions, but also it would not be able to recieve updates. - Maybe these should be templetized and then generated by `setup.py`. Are there any standard template formats we can use? I like the ones in `github.com/smallstep/certificate` - Depending on how you want to integrate multiple chrome clients, the `line-chrome.service` might have to be aliased, i.e. `line-chrome@.service`, and somehow allowed to be controlled by `line.service` user via `sudoer`. ## Slight rant: Currently I don't think there is a stadndard synapse/bridge service infrastructure, and ideally the bridges and plugins should be designed so they can be invoked and editted as something like `matrix-bridges@.service`, but that's a discussion for the core matrix community later on, e.g. if they can provide a `matrix-server.target` that we can later link to add dependencies and so on.
👍 1
lecris added 1 commit 2021-05-18 21:39:19 -04:00
fair commented 2021-05-19 20:53:04 -04:00
Owner
Copy Link

Thanks for this!
Some comments:

  • Regarding User: it's probably enough to have a commented-out User (as you already did) to not break default installs, while still suggesting the best-practice of running with a system user.
  • To handle its dependency on line-chrome.service, maybe line.service can just use Restart=always to make it keep trying to connect to the socket until it succeeds. Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure.
  • Any dependencies on a homeserver unit are IMO out-of-scope for this, given that there's no way to predict which (or how many) homeservers someone has installed, and like you said, there's no standard target/service unit naming scheme for HSs. Plus, this bridge should ideally poll for a HS connection instead of fast-failing, and shouldn't fail or need to restart if the HS crashes (like most bridges), so it may not need a hard dependency on a homeserver at all.
  • Regarding using a socket unit, I was planning on having the node process manage multiple sockets itself, since it looks like most of the connection-management code can already handle multiple connections. Socket units do sound like a good idea though, so I'll think about it more before deciding anything.
Thanks for this! Some comments: * Regarding `User`: it's probably enough to have a commented-out `User` (as you already did) to not break default installs, while still suggesting the best-practice of running with a system user. * To handle its dependency on `line-chrome.service`, maybe `line.service` can just use `Restart=always` to make it keep trying to connect to the socket until it succeeds. Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure. * Any dependencies on a homeserver unit are IMO out-of-scope for this, given that there's no way to predict which (or how many) homeservers someone has installed, and like you said, there's no standard target/service unit naming scheme for HSs. Plus, this bridge should ideally poll for a HS connection instead of fast-failing, and shouldn't fail or need to restart if the HS crashes (like most bridges), so it may not need a hard dependency on a homeserver at all. * Regarding using a socket unit, I was planning on having the node process manage multiple sockets itself, since it looks like most of the connection-management code can already handle multiple connections. Socket units do sound like a good idea though, so I'll think about it more before deciding anything.
👍 1
lecris commented 2021-05-19 21:56:46 -04:00
Author
Contributor
Copy Link

Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure.

That would indeed be preferred. A n time retry with a few seconds time-out should do. If possible I would prefer to avoid Restart=always because it might hide some underlying errors down the line.

Any dependencies on a homeserver unit are IMO out-of-scope for this

Agree, would be nice to have something standardized from the matrix dev team, but until then it is not necessary. Also it should be able to be run with external server in the future like via Dimension or the other pluggable method, so I steadily come to think that it is uneccessary, at least for bridges and such.

Regarding using a socket unit

If node could manage that it would be ideal. One problem with using systemd socket is that it has to be fast to start and persistent. Probably the only place it makes sense to have is on the line.service, but the effect there would be minimal?

Also how about the locations? It would be easy to create these from a template when setting up the database.

> Or better yet, the Python module itself can poll for a socket connection instead of erroring out on first failure. That would indeed be preferred. A n time retry with a few seconds time-out should do. If possible I would prefer to avoid `Restart=always` because it might hide some underlying errors down the line. > Any dependencies on a homeserver unit are IMO out-of-scope for this Agree, would be nice to have something standardized from the matrix dev team, but until then it is not necessary. Also it should be able to be run with external server in the future like via `Dimension` or the other pluggable method, so I steadily come to think that it is uneccessary, at least for bridges and such. > Regarding using a socket unit If node could manage that it would be ideal. One problem with using systemd socket is that it has to be fast to start and persistent. Probably the only place it makes sense to have is on the `line.service`, but the effect there would be minimal? Also how about the locations? It would be easy to create these from a template when setting up the database.
lecris commented 2021-06-03 21:09:09 -04:00
Author
Contributor
Copy Link

Should check github.com/matrix-org/synapse#9803 for seucrity hardening tips and tools like systemd-analyze security from this blog

Should check [github.com/matrix-org/synapse#9803](https://github.com/matrix-org/synapse/pull/9803) for seucrity hardening tips and tools like `systemd-analyze security` from [this blog](https://www.ctrl.blog/entry/systemd-service-hardening.html)
lecris added 1 commit 2021-06-13 08:09:45 -04:00
lecris commented 2021-06-13 08:12:40 -04:00
Author
Contributor
Copy Link

Update: with this change, line-chrome is active only when the initialization is finished by node.

There might be a depandancy on systemd is there any check to see if it still works without it? Maybe the sd.notify will just send commands to the void so it might still work.

Update: with this change, `line-chrome` is active only when the initialization is finished by node. There might be a depandancy on `systemd` is there any check to see if it still works without it? Maybe the `sd.notify` will just send commands to the void so it might still work.
lecris added 1 commit 2021-06-13 08:17:25 -04:00
lecris force-pushed systemd from 9c09285045 to ea56f82f51 2021-06-13 08:19:55 -04:00 Compare
lecris added 1 commit 2021-06-13 08:35:26 -04:00
lecris changed title from WIP: Systemd services to Systemd services 2021-06-13 09:17:43 -04:00
lecris added 1 commit 2021-06-13 19:14:00 -04:00
lecris commented 2021-06-13 19:38:16 -04:00
Author
Contributor
Copy Link

The only other options to set are setting Restart options, which I don't know if it should be added atm or later, and hardening the service files, which is quite a slog to do. Will make a separate file in this branch afterwards.

Should be ok to merge like this, but check the yarn settings on your end as well.

The only other options to set are setting `Restart` options, which I don't know if it should be added atm or later, and hardening the service files, which is quite a slog to do. Will make a separate file in this branch afterwards. Should be ok to merge like this, but check the `yarn` settings on your end as well.
lecris added 5 commits 2021-06-22 21:15:40 -04:00
lecris force-pushed systemd from 968832545e to 9d4f2dafac 2021-06-22 21:19:39 -04:00 Compare
lecris force-pushed systemd from 9d4f2dafac to 3c0dd15616 2021-06-22 21:21:53 -04:00 Compare
fair changed target branch from master to systemd 2021-07-13 01:32:10 -04:00
fair commented 2021-07-13 02:44:55 -04:00
Owner
Copy Link

Merged to new systemd branch.

Merged to new `systemd` branch.
fair closed this pull request 2021-07-13 02:44:56 -04:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: fair/matrix-puppeteer-line#13
No description provided.
Delete Branch "systemd"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?