diff --git a/documentation/INCIDENTS.md b/documentation/INCIDENTS.md index 2cf8b4b..d24ce57 100644 --- a/documentation/INCIDENTS.md +++ b/documentation/INCIDENTS.md @@ -1118,99 +1118,3 @@ all updates where doing using the new "idempotent" `playbook_sysupgrade` (since * 2022.10.24 01:44 - 01:46: `gw-core01` => downtime of the accesspoints in the specified timeframe => downtime of `gw-core01` in the specified timeframe - - -025 2022.11.19 04:00 (ANS) | (maintenance) (try to) steer clients into 5 GHz band ---------------------------------------------------------------------------------- - ---- - -_this log entry was added way after doing the acutal work. -Please read it with a grain of salt_ - ---- - -**problem**: -* (if i remember correctly) way more clients in the 2,4 GhZ band than in the 5 GHz band (3/4 to 1/4) - -**solution**: -* halfe the transmit power in the 2,4 GHz band -* increased transmit power in the 5 GHz band by 1 dBm -* implemented by `5017cb5` - -**impact**: -This restarted wifi on all APs at the same time. -Downtime for all clients for a few seconds at 04:00 in the morning. - -**validation**: -One day afterwards it seemed like there where more clients in the 5 GHz band (50/50), but the datarates dropped for most of them. - -**critisism**: -* placement, transmit power and supported bands of the clients impact 5 GHz utilization -* unsure what actually is the problem -* also did not correctly validate for a few days - - -026 2022.11.20 15:30 (ANS) | (maintenance) replace SFP modules --------------------------------------------------------------- - -... - - -027 2022.11.21 02:00 | (maintenance) attach volume to `eae-adp-jump01` for prometheus -------------------------------------------------------------------------------------- - -20 GB volume an vm attached - -reboot: gegen kurz vor 02:00 -vorher noch syspatch - -``` -eae-adp-jump01# rcctl stop prometheus -eae-adp-jump01# rm -r /var/prometheus/* -eae-adp-jump01# sysctl hw.disknames -eae-adp-jump01# fdisk -iy sd1 -eae-adp-jump01# disklabel -E sd1 -> a a -> -> * -> q -eae-adp-jump01# newfs sd1a -eae-adp-jump01# diff -Naur /etc/fstab.20221121 /etc/fstab ---- /etc/fstab.20221121 Sun Jun 26 23:00:39 2022 -+++ /etc/fstab Mon Nov 21 02:01:03 2022 -@@ -8,3 +8,4 @@ - e1c3571d54635852.j /usr/obj ffs rw,nodev,nosuid 1 2 - e1c3571d54635852.i /usr/src ffs rw,nodev,nosuid 1 2 - e1c3571d54635852.e /var ffs rw,nodev,nosuid 1 2 -+a0469c9f38992e1d.a /var/prometheus ffs rw,nodev,nosuid 1 2 -eae-adp-jump01# mount /var/prometheus -eae-adp-jump01# chown _prometheus:_prometheus /var/prometheus -eae-adp-jump01# rcctl start prometheus -``` - -028 2022.11.29 02:00 | periodically restart prometheus -------------- - -028 2022.11.29 03:00 | (maintenance) activate auto start for offloader -------------- - -offloader down from 02:50 to 03:05 -gw-core down from 02:53 to 02:55 - - -029 2022.11.30 15:30 | (maintenance) replace switches ----- - -* 16:34:30 - 16:34:50: `ffl-ans-sw-distribution01` - * quickly replaced device and connections -* 16:49: `ffl-ans-sw-access01`: minimal L2 downtme, accesspoint needed a reboot - * power up new device alongside - * bridge old and new device with short patch cable - * move sfp uplink to new device - * move first ap to new switch - * wait till ap was back up and serving clients - * move second ap - * teardown old device -* 17:09:30 - 17:10:15`: `ffl-ans-sw-access02` - * quickly replaced device and connections