Please note that viewing attachments of hyphae is not supported in history for now. Get Mycomarkup source of this revision

🌻Infrastructure And Operations Inventory

---
title: Inventory
---

Last Updated: March 2021

Overview

Hypha infrastructure_and_operations/inventory/overview.png does not exist

This diagram was created with app.diagrams.net/.
To edit it, download the <a download href="inventory-overview.drawio">diagram file</a> and edit it with the app.diagrams.net/ web application, or you may run the application from [source](github.com/jgraph/drawio) if you wish.

Physical Hosts

- baikal
- 69.61.110.118
- cyberia's first rack server, installed in CyberWurx datacenter in Atlanta Georgia
- j3s is the only authorized support contact right now and the only one who can log into the CyberWurx portal

- dredd
- dynamic(ish) ip address
- NOTE: dredd uses port 3217 for ssh. connect with ssh -p 3217 dredd.cyberia.club
- olde desktop located in j3s's house

- magnataur
- dynamic(ish) ip address
- NOTE: magnataur uses port 3216 for ssh. connect with ssh -p 3216 magnataur.cyberia.club
- olde desktop located in j3s's house

Cloud Service Accounts

- namecheap
- fack's namecheap account is currently being used to manage DNS entires for:
- cyberia.club
- nullhex.com
- capsul.org
- all DNS updates are being done manually by j3s.
- conventions:
- A records are named after hostnames & point to VMs / physical hosts
- CNAMEs are named after the service & point to the A record of the host the service runs on

- CyberWurx portal
- Allows us to add reverse DNS entries for Capsuls
- View metrics, get datacenter information, support tickets, etc
- Right now j3s is the only one who can log in / be authorized for support. Can add others though!

Capsul

Most of cyberia's services run on [Capsul](capsul.org), our Virtual Machine Management tool & service.

Ansible Managed Capsuls:
capsul-ay3yh10q2q  f1-xs  69.61.2.230  alpine311  Jun 20 2020 domechild.cyberia.club  (email server)
capsul-c04bbf593b  f1-s   69.61.2.246  alpine311  Jun 01 2020 raaz.cyberia.club       (NSHC / North Star Health Collective) 
capsul-pfgy2tthx9  f1-xs  69.61.2.167  alpine311  May 10 2020 legion.cyberia.club     (postgres for forge & others in the future)
capsul-id502edkg0  f1-xs  69.61.2.170  alpine311  Apr 01 2020 rosewater.cyberia.club  (cyberia forge)
capsul-t6tfb2dh5p  f1-m   69.61.2.183  alpine311  May 10 2020 mothership.cyberia.club (prometheus & grafana & future logg agg)
capsul-w6hsx09r7v  f1-xs  69.61.2.213  alpine311  Aug 20 2020 leckie.cyberia.club     (ansible bastion + build submitter)
capsul-f6crtfzx5c  f1-xs  69.61.2.218  alpine313  Mar 01 2021 comet.cyberia.club      (owncast server)
capsul-e1tfrw0637  f1-xs  69.61.2.201  alpine313  Mar 13 2021 kindred.cyberia.club    (mastodon server)

Legacy Capsuls:
capsul-yi9ffqbjly  f1-x   69.61.2.188  debian10   Apr 15 2020 btcpay.cyberia.club     (btcpay) 
cvm-lqj2x9nxic	   f1-l   69.61.2.190  debian10   Mar 07 2020 matrix.cyberia.club     (cyberia matrix) 
cvm-m1tjv0lljd	   f1-xs  69.61.2.178  debian10   Mar 10 2020 elliot.cyberia.club     (websites & git.cyberia.club, nullhex.com)

The Ansible Managed servers should have a user account for each user. The Legacy servers & baikal only have one user named cyberian, with everyone's keys authorized for that server.

Contact j3s, forest, or vvesley for more information on cyberia's capsul account.

Host Key Fingerprints

NOTE: you can control what kind of host key your ssh client will use like this:

ssh -o HostKeyAlgorithms=ssh-ed25519 example.cyberia.club

baikal.cyberia.club
  ECDSA    SHA256:85GTFfUpDDefcNcIROtFpuTiHC1j3iNU74aaKFO03+0
  ED25519  SHA256:v9MEa97wnmA75CyzQC5lW8nOI56LJ4jTmD2f68udK80

magnataur.cyberia.club
  NOTE:    magnataur uses port 3216 for ssh. connect with ssh -p 3216 magnataur.cyberia.club
  ECDSA    SHA256:kPOBn03CH176zrTlFDVmjFJpWi1OGHhkNCiK6stNn/0
  ED25519  SHA256:7M8ppVJ534Axz1ZXt6NheBxYkqY9UJ3AAmb9BmY9bYk

dredd.cyberia.club
  NOTE:    dredd uses port 3217 for ssh. connect with ssh -p 3217 dredd.cyberia.club
  ECDSA    SHA256:5157aYG7PT8Y0I4sTzlpQ5i/E3bq4aPF9T1P+xj+l9Q
  ED25519  SHA256:w6F0NXBoLCXG60yXoI3QhYGiLlPCr6YrK/OUSSDcmAw

mothership.cyberia.club
  ECDSA    SHA256:3XJG2fyaPDJWjnEOW3q2KiWg5qLV6hmEPczvp8GqhE0
  ED25519  SHA256:njIT2k1t6hHuOO0VjBNmHW1QSGN4GEqQQMj/BGpnBa0

domechild.cyberia.club
  ECDSA    SHA256:IQqTPv14u3dG62hS0q2Mr6pef6KwpjPKM2uVP+SK+qA
  ED25519  SHA256:3z5BI2ZEZjzDEh0B7a2GxgMa4faqA3Y6bQdGcQp4G88

rosewater.cyberia.club
  ECDSA    SHA256:dAbABreDUpV9AG7kChcx9S6+6f+fmnhqwwInqYoxcwU
  ED25519  SHA256:nT+ISIGV95MBKkIpcHTKo30lx4qRQ0Cpu1iM3w6+Sh0

legion.cyberia.club
  ECDSA    SHA256:EW9ydcgLg/pwoA0GPsI0VVeIBpnSi7aIHhvXOQBa+Xg
  ED25519  SHA256:cWLBFESOHrmVFrLRLjxrY4tcPmVRerJe1SB/+6tXSxAv

leckie.cyberia.club
  ECDSA    SHA256:KbzxzEKP21B0S3A/SKqqGmjiymnkk7byvoc6W4SxEwM
  ED25519  SHA256:M1QPflfIrsbhVlMaomvGQsr5AZS5YRkBHv+pnyI7bg4

raaz.cyberia.club
  ECDSA    SHA256:AJb0bZN2PTTm83zf5zI1IOEIVfeXUxQl/vTode/88jA
  ED25519  SHA256:zJv6E6lG4dAsqNmDHTO/qFVlTESKYq/KD29e8Nt/6j4

matrix.cyberia.club
  ECDSA    SHA256:VlRPAqLGxY4JUVhYirOVlfuDFtgTbaiw3x29xYizEeU
  ED25519  SHA256:BExhsVPNTp49jyJ6ezRf+Nn4TxPj8D9VZMhnjMABq6g

elliot.cyberia.club
  ECDSA    SHA256:/tsASDZ+MX519DC/Y7mHbV2CYCPnyMAbX1e0GHBOin0
  ED25519  SHA256:B9QNCnz57agsI40tMVU8UwyvZqMbz/p1ZNH5E1gL3io

btcpay.cyberia.club
  ECDSA    SHA256:CdqdUvG0Obfdq9kkeQSETVhSJO2oCAdEAjDCydQWcDI
  ED25519  SHA256:WcjrJtvev3+rAu98TFGJoxx/CytLCg+GfEXBMVOl5Hw

comet.cyberia.club
  ECDSA    SHA256:UcDUCFd/U3F8ECG/RKxLbJRAAiMBSRKVKqDM0hmjwJ8
  ED25519  SHA256:SoOuSzKmpUd4x8Y8G32EAfQTY15agz1z7zJJCWdI8Tw

kindred.cyberia.club
  ECDSA    SHA256:M2oWKPgOqynag2nXrxnideac+r4Vb2tAsEz5ddEh/EM
  ED25519  SHA256:wCyMJYgoPAwlFKTXw41v/q8kypuand4fmhY4zsWdGlc

Automation (Ansible)

The [Ops Handbook](git.cyberia.club/services/ops-handbook/) is still on the old git server, it is the main repo with the ansible inventory & playbooks.

Ansible bastion host/automation is on leckie.cyberia.club

Service Inventory

| User-oriented Name | URL | Developer-oriented Name | Host | Deployment Code | Application Code |
|-----------------------|-------------------------|-------------------------|-----------------------|--------------------|-------------------|
|cyberia's matrix server|matrix.cyberia.club/| synapse | matrix.cyberia.club | [ansible/roles/synapse](git.cyberia.club/services/ops-handbook/tree/ansible/roles/synapse) | [matrix-org/synapse](github.com/matrix-org/synapse) |
|cyberia's matrix server|riot.cyberia.club/| element (used to be called riot) | matrix.cyberia.club | [ansible/roles/riot](git.cyberia.club/services/ops-handbook/tree/ansible/roles/riot) | [vector-im/element-web](github.com/vector-im/element-web) |
|cyberia's matrix server| N/A | postgres | matrix.cyberia.club | [ansible/roles/postgresql](git.cyberia.club/services/ops-handbook/tree/ansible/roles/postgresql) | [git.postgresql.org](git.postgresql.org/git/postgresql.git) |
|cyberia's matrix server| N/A | irc bridge to freenode | matrix.cyberia.club | TBD | [matrix-org/matrix-appservice-irc](github.com/matrix-org/matrix-appservice-irc) |
|cyberia's matrix server| matrix.cyberia.club/_synapse/metrics | matrix prometheus exporter | matrix.cyberia.club | TBD | [matrix-org/synapse/metrics](github.com/matrix-org/synapse/tree/develop/synapse/metrics) |
|nullhex email | nullhex.com/ | alps | elliot.cyberia.club | TBD | [~emersion/alps/](sr.ht/~emersion/alps/) |
|nullhex email | nullhex.com ports 25 & 587 (STARTTLS) | opensmtpd | domechild.cyberia.club | [ansible/roles/opensmtpd](git.cyberia.club/services/ops-handbook/tree/ansible/roles/opensmtpd) | [OpenSMTPD/OpenSMTPD](github.com/OpenSMTPD/OpenSMTPD/) |
|nullhex email | nullhex.com:993 (imap) | dovecot | domechild.cyberia.club | [ansible/roles/dovecot](git.cyberia.club/services/ops-handbook/tree/ansible/roles/dovecot) | [dovecot/core](github.com/dovecot/core) |
|nullhex email | N/A | rspamd | domechild.cyberia.club | TBD | [rspamd/rspamd](github.com/rspamd/rspamd) |
| capsul | capsul.org | capsul | baikal.cyberia.club | TBD | [~forest/capsul-flask/](giit.cyberia.club/~forest/capsul-flask/) |
| forge (cyberia's git server) |forge.cyberia.club/ | sourcehut | rosewater.cyberia.club | [see the ops-handbook](git.cyberia.club/services/ops-handbook/tree/docs/forge.md) | [~sircmpwn/sourcehut](sr.ht/~sircmpwn/sourcehut/) |
| forge (cyberia's git server) |N/A | postgres | legion.cyberia.club | TBD | [git.postgresql.org](git.postgresql.org/git/postgresql.git) |
| concourse (the new build server) | concourse.cyberia.club/ | concourse | rosewater.cyberia.club | TBD | [concourse/concourse](github.com/concourse/concourse) |
| vault (build secrets manager) | N/A | vault | rosewater.cyberia.club | [ansible/roles/concourse-vault](git.cyberia.club/services/ops-handbook/tree/ansible/roles/concourse-vault/files) | [hashicorp/vault](github.com/hashicorp/vault) |
| cyberia's website | cyberia.club/ | nginx static site | elliot.cyberia.club | TBD | [services/website](git.cyberia.club/services/website/) |
| the old git server | git.cyberia.club/ | cgit | elliot.cyberia.club | TBD | [git.zx2c4.com/cgit](git.zx2c4.com/cgit/) |
| prometheus | prometheus.cyberia.club/ | prometheus | mothership.cyberia.club | [rules](git.cyberia.club/services/ops-handbook/tree/rules) & [ansible/roles/prometheus](git.cyberia.club/services/ops-handbook/tree/ansible/roles/prometheus) | [prometheus/prometheus](github.com/prometheus/prometheus) |
| alertmanager | N/A | alertmanager | mothership.cyberia.club | same as prometheus | [prometheus/alertmanager](github.com/prometheus/alertmanager) |
| grafana | grafana.cyberia.club/ | grafana | mothership.cyberia.club | [ansible/roles/grafana](git.cyberia.club/services/ops-handbook/tree/ansible/roles/grafana) | [grafana/grafana](github.com/grafana/grafana) |
| Jackal | bot.j3s.sh | go-neb (matrix bot) | mothership.cyberia.club | TBD | [matrix-org/go-neb (forest's fork)](giit.cyberia.club/~forest/go-neb/log/forest-feature-rebase-2) |
| Stream | stream.cyberia.club | owncast | comet.cyberia.club | TBD | [owncast/owncast](github.com/owncast/owncast) |
| Mastodon | social.cyberia.club | hometown | kindred.cyberia.club | TBD | [hometown-fork/hometown](github.com/hometown-fork/hometown) |

LetsEncrypt Certificate Inventory

For information on certificates which are managed by uacme automatically, see
git.cyberia.club/services/ops-handbook/tree/ansible/hosts
and the tls_certs variable in git.cyberia.club/services/ops-handbook/tree/ansible/group_vars

Certificates which are exceptions to the rule:

btcpay.cyberia.club
 - btcpay.cyberia.club certificate is automatically managed by btcpayserver-docker

elliot.cyberia.club
The following are managed by a script called acme.sh located at `/root/.acme.sh/`
 - capsul.org
 - www.capsul.org
 - nullhex.com
 - cyberia.club
 - git.cyberia.club

matrix.cyberia.club
The following are managed by a script called acme.sh located at `/root/.acme.sh/`
  - matrix.cyberia.club
  - riot.cyberia.club

magnataur.cyberia.club
The following are managed by Caddy on the router which sits in front of magnataur. Ask j3s or fack about this.
  - cafe.cyberia.club
  - mumble.cyberia.club

How to use acme.sh:

systemctl stop nginx ; acme.sh --renew --domain git.cyberia.club; systemctl start nginx ;

If you get an error like

Please specify at least one validation method: '--webroot', '--standalone', '--apache', '--nginx' or '--dns' etc.

Then you must edit the config file, for example

nano root@elliot:~/.acme.sh# nano cyberia.club/cyberia.club.conf

and change Le_Webroot='' to Le_Webroot='no' inside the <domain-name>/<domain-name.conf> file. [see: github issue](github.com/acmesh-official/acme.sh/issues/1172)

certificate expiry alerts

The certificate expiry alerts are defined here: git.cyberia.club/services/ops-handbook/tree/rules/alerts.yml#n112

The probe_ssl_earliest_cert_expiry metric is written by the blackbox exporter, configured here: git.cyberia.club/services/ops-handbook/tree/ansible/roles/prometheus/templates/prometheus.yml.j2#n82

Notes

poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/

🌻:Infrastructure And Operations/Inventory