---
title: Inventory
---

Last Updated: March 2021

## Overview


img {
   infrastructure_and_operations/inventory/inventory-overview.png
}


This diagram was created with https://app.diagrams.net/.
To edit it, download the <a download href="inventory-overview.drawio">diagram file</a> and edit it with the https://app.diagrams.net/ web application, or you may run the application from [source](https://github.com/jgraph/drawio) if you wish.

## Physical Hosts

 - baikal
   - 69.61.110.118
   - cyberia's first rack server, installed in  CyberWurx datacenter in Atlanta Georgia
   - j3s is the only authorized support contact right now and the only one who can log into the CyberWurx portal

 - dredd
   - dynamic(ish) ip address
   - NOTE: dredd uses port 3217 for ssh. connect with `ssh -p 3217 dredd.cyberia.club`
   - olde desktop located in j3s's house

 - magnataur
   - dynamic(ish) ip address
   - NOTE: magnataur uses port 3216 for ssh. connect with `ssh -p 3216 magnataur.cyberia.club`
   - olde desktop located in j3s's house

## Cloud Service Accounts

  - namecheap
    - fack's namecheap account is currently being used to manage DNS entires for:
      - cyberia.club
      - nullhex.com
      - capsul.org
    - all DNS updates are being done manually by j3s.
    - conventions:
      - A records are named after hostnames & point to VMs / physical hosts
      - CNAMEs are named after the service & point to the A record of the host the service runs on

  - CyberWurx portal
    - Allows us to add reverse DNS entries for Capsuls
    - View metrics, get datacenter information, support tickets, etc
    - Right now j3s is the only one who can log in / be authorized for support. Can add others though!

## Capsul

Most of cyberia's services run on [Capsul](https://capsul.org), our Virtual Machine Management tool & service.

```
Ansible Managed Capsuls:
capsul-ay3yh10q2q  f1-xs  69.61.2.230  alpine311  Jun 20 2020 domechild.cyberia.club  (email server)
capsul-c04bbf593b  f1-s   69.61.2.246  alpine311  Jun 01 2020 raaz.cyberia.club       (NSHC / North Star Health Collective) 
capsul-pfgy2tthx9  f1-xs  69.61.2.167  alpine311  May 10 2020 legion.cyberia.club     (postgres for forge & others in the future)
capsul-id502edkg0  f1-xs  69.61.2.170  alpine311  Apr 01 2020 rosewater.cyberia.club  (cyberia forge)
capsul-t6tfb2dh5p  f1-m   69.61.2.183  alpine311  May 10 2020 mothership.cyberia.club (prometheus & grafana & future logg agg)
capsul-w6hsx09r7v  f1-xs  69.61.2.213  alpine311  Aug 20 2020 leckie.cyberia.club     (ansible bastion + build submitter)
capsul-f6crtfzx5c  f1-xs  69.61.2.218  alpine313  Mar 01 2021 comet.cyberia.club      (owncast server)
capsul-e1tfrw0637  f1-xs  69.61.2.201  alpine313  Mar 13 2021 kindred.cyberia.club    (mastodon server)

Legacy Capsuls:
capsul-yi9ffqbjly  f1-x   69.61.2.188  debian10   Apr 15 2020 btcpay.cyberia.club     (btcpay) 
cvm-lqj2x9nxic	   f1-l   69.61.2.190  debian10   Mar 07 2020 matrix.cyberia.club     (cyberia matrix) 
cvm-m1tjv0lljd	   f1-xs  69.61.2.178  debian10   Mar 10 2020 elliot.cyberia.club     (websites & git.cyberia.club, nullhex.com)
```

The Ansible Managed servers should have a user account for each user. The Legacy servers & baikal only have one user named cyberian, with everyone's keys authorized for that server.

Contact j3s, forest, or vvesley for more information on cyberia's capsul account.

#### Host Key Fingerprints

**NOTE:** you can control what kind of host key your ssh client will use like this: 

`ssh -o HostKeyAlgorithms=ssh-ed25519 example.cyberia.club`

```
baikal.cyberia.club
  ECDSA    SHA256:85GTFfUpDDefcNcIROtFpuTiHC1j3iNU74aaKFO03+0
  ED25519  SHA256:v9MEa97wnmA75CyzQC5lW8nOI56LJ4jTmD2f68udK80

magnataur.cyberia.club
  NOTE:    magnataur uses port 3216 for ssh. connect with ssh -p 3216 magnataur.cyberia.club
  ECDSA    SHA256:kPOBn03CH176zrTlFDVmjFJpWi1OGHhkNCiK6stNn/0
  ED25519  SHA256:7M8ppVJ534Axz1ZXt6NheBxYkqY9UJ3AAmb9BmY9bYk

dredd.cyberia.club
  NOTE:    dredd uses port 3217 for ssh. connect with ssh -p 3217 dredd.cyberia.club
  ECDSA    SHA256:5157aYG7PT8Y0I4sTzlpQ5i/E3bq4aPF9T1P+xj+l9Q
  ED25519  SHA256:w6F0NXBoLCXG60yXoI3QhYGiLlPCr6YrK/OUSSDcmAw

mothership.cyberia.club
  ECDSA    SHA256:3XJG2fyaPDJWjnEOW3q2KiWg5qLV6hmEPczvp8GqhE0
  ED25519  SHA256:njIT2k1t6hHuOO0VjBNmHW1QSGN4GEqQQMj/BGpnBa0

domechild.cyberia.club
  ECDSA    SHA256:IQqTPv14u3dG62hS0q2Mr6pef6KwpjPKM2uVP+SK+qA
  ED25519  SHA256:3z5BI2ZEZjzDEh0B7a2GxgMa4faqA3Y6bQdGcQp4G88

rosewater.cyberia.club
  ECDSA    SHA256:dAbABreDUpV9AG7kChcx9S6+6f+fmnhqwwInqYoxcwU
  ED25519  SHA256:nT+ISIGV95MBKkIpcHTKo30lx4qRQ0Cpu1iM3w6+Sh0

legion.cyberia.club
  ECDSA    SHA256:EW9ydcgLg/pwoA0GPsI0VVeIBpnSi7aIHhvXOQBa+Xg
  ED25519  SHA256:cWLBFESOHrmVFrLRLjxrY4tcPmVRerJe1SB/+6tXSxAv

leckie.cyberia.club
  ECDSA    SHA256:KbzxzEKP21B0S3A/SKqqGmjiymnkk7byvoc6W4SxEwM
  ED25519  SHA256:M1QPflfIrsbhVlMaomvGQsr5AZS5YRkBHv+pnyI7bg4

raaz.cyberia.club
  ECDSA    SHA256:AJb0bZN2PTTm83zf5zI1IOEIVfeXUxQl/vTode/88jA
  ED25519  SHA256:zJv6E6lG4dAsqNmDHTO/qFVlTESKYq/KD29e8Nt/6j4

matrix.cyberia.club
  ECDSA    SHA256:VlRPAqLGxY4JUVhYirOVlfuDFtgTbaiw3x29xYizEeU
  ED25519  SHA256:BExhsVPNTp49jyJ6ezRf+Nn4TxPj8D9VZMhnjMABq6g

elliot.cyberia.club
  ECDSA    SHA256:/tsASDZ+MX519DC/Y7mHbV2CYCPnyMAbX1e0GHBOin0
  ED25519  SHA256:B9QNCnz57agsI40tMVU8UwyvZqMbz/p1ZNH5E1gL3io

btcpay.cyberia.club
  ECDSA    SHA256:CdqdUvG0Obfdq9kkeQSETVhSJO2oCAdEAjDCydQWcDI
  ED25519  SHA256:WcjrJtvev3+rAu98TFGJoxx/CytLCg+GfEXBMVOl5Hw

comet.cyberia.club
  ECDSA    SHA256:UcDUCFd/U3F8ECG/RKxLbJRAAiMBSRKVKqDM0hmjwJ8
  ED25519  SHA256:SoOuSzKmpUd4x8Y8G32EAfQTY15agz1z7zJJCWdI8Tw

kindred.cyberia.club
  ECDSA    SHA256:M2oWKPgOqynag2nXrxnideac+r4Vb2tAsEz5ddEh/EM
  ED25519  SHA256:wCyMJYgoPAwlFKTXw41v/q8kypuand4fmhY4zsWdGlc
```


## Automation (Ansible)

The [Ops Handbook](https://git.cyberia.club/services/ops-handbook/) is still on the old git server, it is the main repo with the ansible inventory & playbooks.

Ansible bastion host/automation is on leckie.cyberia.club

## Service Inventory

|  User-oriented Name   |         URL             | Developer-oriented Name |        Host           |    Deployment Code |  Application Code |
|-----------------------|-------------------------|-------------------------|-----------------------|--------------------|-------------------|
|cyberia's matrix server|https://matrix.cyberia.club/|         synapse         |  matrix.cyberia.club  | [ansible/roles/synapse](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/synapse) | [matrix-org/synapse](https://github.com/matrix-org/synapse) |
|cyberia's matrix server|https://riot.cyberia.club/| element (used to be called riot) |  matrix.cyberia.club  | [ansible/roles/riot](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/riot) | [vector-im/element-web](https://github.com/vector-im/element-web) |
|cyberia's matrix server| N/A                       |  postgres |  matrix.cyberia.club  | [ansible/roles/postgresql](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/postgresql) |  [git.postgresql.org](https://git.postgresql.org/git/postgresql.git) |
|cyberia's matrix server| N/A                       |  irc bridge to freenode    |  matrix.cyberia.club  |     TBD    | [matrix-org/matrix-appservice-irc](https://github.com/matrix-org/matrix-appservice-irc) |
|cyberia's matrix server| https://matrix.cyberia.club/_synapse/metrics | matrix prometheus exporter |  matrix.cyberia.club  |     TBD    | [matrix-org/synapse/metrics](https://github.com/matrix-org/synapse/tree/develop/synapse/metrics) |
|nullhex email          | https://nullhex.com/      | alps |  elliot.cyberia.club  |     TBD    | [~emersion/alps/](https://sr.ht/~emersion/alps/) |
|nullhex email          | nullhex.com ports 25 & 587 (STARTTLS) | opensmtpd |  domechild.cyberia.club  |     [ansible/roles/opensmtpd](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/opensmtpd)    | [OpenSMTPD/OpenSMTPD](https://github.com/OpenSMTPD/OpenSMTPD/) |
|nullhex email          | nullhex.com:993 (imap)   | dovecot |  domechild.cyberia.club  |     [ansible/roles/dovecot](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/dovecot)    | [dovecot/core](https://github.com/dovecot/core) |
|nullhex email          | N/A                       | rspamd  |  domechild.cyberia.club  |     TBD    | [rspamd/rspamd](https://github.com/rspamd/rspamd) |
| capsul                | https://capsul.org       | capsul | baikal.cyberia.club | TBD       | [~forest/capsul-flask/](https://giit.cyberia.club/~forest/capsul-flask/) |
| forge (cyberia's git server) |https://forge.cyberia.club/ | sourcehut | rosewater.cyberia.club |      [see the ops-handbook](https://git.cyberia.club/services/ops-handbook/tree/docs/forge.md)       | [~sircmpwn/sourcehut](https://sr.ht/~sircmpwn/sourcehut/) |
| forge (cyberia's git server) |N/A                | postgres | legion.cyberia.club |     TBD    | [git.postgresql.org](https://git.postgresql.org/git/postgresql.git) |
| concourse (the new build server) | https://concourse.cyberia.club/ | concourse | rosewater.cyberia.club | TBD | [concourse/concourse](https://github.com/concourse/concourse) |
| vault (build secrets manager) | N/A | vault | rosewater.cyberia.club | [ansible/roles/concourse-vault](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/concourse-vault/files) | [hashicorp/vault](https://github.com/hashicorp/vault) |
| cyberia's website     | https://cyberia.club/    | nginx static site   | elliot.cyberia.club |     TBD    | [services/website](https://git.cyberia.club/services/website/) |
| the old git server    | https://git.cyberia.club/ | cgit   | elliot.cyberia.club |     TBD    | [git.zx2c4.com/cgit](https://git.zx2c4.com/cgit/) |
| prometheus   | https://prometheus.cyberia.club/ | prometheus   | mothership.cyberia.club | [rules](https://git.cyberia.club/services/ops-handbook/tree/rules) & [ansible/roles/prometheus](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/prometheus) | [prometheus/prometheus](https://github.com/prometheus/prometheus) |
| alertmanager   | N/A                            | alertmanager   | mothership.cyberia.club | same as prometheus | [prometheus/alertmanager](https://github.com/prometheus/alertmanager) |
| grafana          | https://grafana.cyberia.club/ | grafana   | mothership.cyberia.club | [ansible/roles/grafana](https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/grafana) | [grafana/grafana](https://github.com/grafana/grafana) |
| Jackal           | https://bot.j3s.sh            |   go-neb (matrix bot)  | mothership.cyberia.club | TBD | [matrix-org/go-neb (forest's fork)](https://giit.cyberia.club/~forest/go-neb/log/forest-feature-rebase-2) |
| Stream           | https://stream.cyberia.club            | owncast   | comet.cyberia.club | TBD | [owncast/owncast](https://github.com/owncast/owncast) |
| Mastodon           | https://social.cyberia.club            | hometown   | kindred.cyberia.club | TBD | [hometown-fork/hometown](https://github.com/hometown-fork/hometown) |


## LetsEncrypt Certificate Inventory

For information on certificates which are managed by uacme automatically, see 
https://git.cyberia.club/services/ops-handbook/tree/ansible/hosts
and the `tls_certs` variable in https://git.cyberia.club/services/ops-handbook/tree/ansible/group_vars

Certificates which are exceptions to the rule: 

```
btcpay.cyberia.club
 - btcpay.cyberia.club certificate is automatically managed by btcpayserver-docker

elliot.cyberia.club
The following are managed by a script called acme.sh located at `/root/.acme.sh/`
 - capsul.org
 - www.capsul.org
 - nullhex.com
 - cyberia.club
 - git.cyberia.club

matrix.cyberia.club
The following are managed by a script called acme.sh located at `/root/.acme.sh/`
  - matrix.cyberia.club
  - riot.cyberia.club

magnataur.cyberia.club
The following are managed by Caddy on the router which sits in front of magnataur. Ask j3s or fack about this.
  - cafe.cyberia.club
  - mumble.cyberia.club
```

### How to use acme.sh:

```
systemctl stop nginx ; acme.sh --renew --domain git.cyberia.club; systemctl start nginx ;
```

If you get an error like 
```
Please specify at least one validation method: '--webroot', '--standalone', '--apache', '--nginx' or '--dns' etc.
```

Then you must edit the config file, for example 

```
nano root@elliot:~/.acme.sh# nano cyberia.club/cyberia.club.conf
```

and change `Le_Webroot=''` to `Le_Webroot='no'`  inside the `<domain-name>/<domain-name.conf>` file.  [see: github issue](https://github.com/acmesh-official/acme.sh/issues/1172)


### certificate expiry alerts

The certificate expiry alerts are defined here: https://git.cyberia.club/services/ops-handbook/tree/rules/alerts.yml#n112

The `probe_ssl_earliest_cert_expiry` metric is written by the blackbox exporter, configured here: https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/prometheus/templates/prometheus.yml.j2#n82

## Notes

https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/