gluon-firmware/docs/releases/v2018.1.rst

Gluon 2018.1
############

Important notes
***************

This version changes the flash partition layout on some devices (TP-Link CPE/WBS 210/510). To avoid
upgrade failures, make sure to upgrade to Gluon 2017.1.8 or the latest Gluon 2016.2.x (unreleased)
before installing Gluon 2018.1.

Some of the following paragraphs describe so-called "feature flags". This new concept is
explained in :ref:`user-site-feature-flags`.

Added hardware support
**********************

ar71xx-generic
==============

* ALFA NETWORK

  - AP121F

* AVM

  - FRITZ!Box 4020

* OpenMesh

  - A40
  - A60
  - OM2P v4
  - OM2P-HS v4

* TP-Link

  - Archer C59 v1 [#noibss]_
  - CPE210 v2

ar71xx-nand
===========

* ZyXEL

  - NBG6716

ar71xx-tiny
===========

* TP-Link

  - TL-WA901ND v5

ipq806x [#newtarget]_ [#noibss]_
================================

* TP-Link

  - Archer C2600

ramips-mt7620 [#newtarget]_ [#noibss]_
======================================

* GL Innovations

  - GL-MT300A
  - GL-MT300N
  - GL-MT750

ramips-mt7628 [#newtarget]_ [#noibss]_
======================================

* VoCore

  - VoCore 2

ramips-rt305x [#newtarget]_ [#noibss]_
======================================

* A5

  - V11

* D-Link

  - DIR615 (D1, D2, D3, D4, H1)

* VoCore

  - VoCore (8MB, 16MB)

sunxi [#newtarget]_
===================

* LeMaker/SinoVoip

  - Banana Pi (M1)


.. [#newtarget]
  New target

.. [#noibss]
  Device or target does not support AP+IBSS mode: This device or target will not be built
  when *GLUON_WLAN_MESH* is set to ``ibss``.

New features
************

Multidomain support
===================

When mesh networks grow too large, it becomes necessary to split them into
multiple independent mesh domains to allow the meshes to work with reasonable
performance. Formerly, the only way to achieve this with Gluon was to build
a separate set of firmware images for each domain.

With Gluon 2018.1, multidomain firmwares can be used to achieve the same,
using only a single site configuration that is basis for several different
domain-specific configurations. The feature is explained in detail in
:doc:`../features/multidomain`.

Wired mesh encapsulation
========================

Gluon now supports encapsulating wired mesh traffic (Mesh on LAN/WAN) in
`VXLAN <https://en.wikipedia.org/wiki/Virtual_Extensible_LAN>`_.
See :doc:`../features/wired-mesh` for details on this feature.

Router advertisement filtering
==============================

Similar to the builtin batman-adv gateway feature for IPv4, the *gluon-radv-filterd* package
(*radv-filterd* feature flag) allows to filter IPv6 router advertisements received from the mesh
so that only the RAs with the best routing metric (TQ) reach the clients, ensuring that
the "best" (topologically closest) gateway is chosen as the IPv6 default route, thereby
reducing gateway crosstalk.

At the moment, this feature only filters RAs forwarded to clients; the RAs handled on
the nodes themselves will be unfiltered, so the nodes will still use arbitrary default
gateways.

IGMP/MLD segmentation
=====================

The IGMP/MLD segmentation feature previously provided by the *gluon-ebtables-segment-mld*
package has been extended and moved into the Gluon core; it does not exist as a separate package
anymore.

Filtering IGMP/MLD queries directed towards the mesh ensures that each node becomes the multicast querier
for its own clients (unless there are other multicast-aware switches connected to the node), rather
than electing a single, basically arbitrary node in the mesh to become the querier. Overall,
this should significantly improve the reliability of multicast in the mesh. This is especially
important for IPv6, as the IPv6 Neighbour Discovery Protocol (NDP) is based on local multicast.

See also the documentation of the :ref:`site.conf mesh section <user-site-mesh>`.

gluon-ebtables-limit-arp
========================

The *gluon-ebtables-limit-arp* (*ebtables-limit-arp* feature flag) package adds filters to limit the
rate of ARP requests client devices are allowed to send into the mesh.

Certain client applications are known to generate a significant amount of such ARP requests and
are reportedly becoming more and more common. Without this package, such clients are one
known cause for mesh wide load and congestion problems (see also the :ref:`releases-v2018.1-known-issues`
section below).

Because of this package's implementation, which relies on frequent dynamic updates
- something ebtables does not perform well at - it is not included by default, as it can
cause unnecessary load. Feedback, especially with a close look on load and congestion on
nodes with a large number of changing client devices, is very much welcome. Depending on the
feedback, we might enable this feature by default in a future release.

Public key in respondd data (optional)
======================================

If desired, the fastd public key of a node can be included in the respondd nodeinfo data,
facilitating the correlations of VPN peers and nodes. As the VPN key is transmitted unencrypted
in the fastd handshake, this would theoretically allow an ISP to determine which nodes
are operated behind which internet line. Therefore, this feature must be enabled explicitly
by setting *mesh_vpn.pubkey_privacy* to ``false`` in *site.conf*.

B.A.T.M.A.N. V (experimental)
=============================

When using batman-adv compat 15, it is now possible to switch to the new routing
algorithm B.A.T.M.A.N. V (while the old algorithm is called B.A.T.M.A.N. IV) by
setting *mesh.batman_adv.routing_algo* to ``"BATMAN_V"``. Note that the new routing
algorithm is not backwards-compatible, so nodes using different algorithms can
not interoperate.

.. _releases-v2018.1-site-changes:

Site changes
************

site.mk
=======

* Due to improved package dependency handling, the packages
  *gluon-config-mode-core* and *gluon-setup-mode* do not need
  to be listed explicitly in *site.mk* anymore; they will be
  pulled in implicitly.
* Including the *ebtables-limit-arp* feature flag is recommended. Please note
  the abovementioned caveats on this feature.
* We recommend to use *GLUON_FEATURES* for all Gluon packages, and rely on
  *GLUON_SITE_PACKAGES* for non-Gluon (OpenWrt) packages only, as explained
  in :ref:`user-site-feature-flags`.
* The `GLUON_ATH10K_MESH` variable was renamed to `GLUON_WLAN_MESH`.

site.conf
=========

When updating a site configuration from Gluon 2017.1.x, the following changes
must be made:

* .. code-block:: lua

    domain_seed = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',

  These 32 bytes of random data (encoded in hexadecimal) are used to seed a number
  of site/domain specific random values that must be the same on all nodes of the
  same mesh, but different for different meshes. The following command
  can be used to generate such a random value:

  .. code-block:: shell

    echo $(hexdump -v -n 32 -e '1/1 "%02x"' </dev/urandom)

  In multidomain setups, repeat this command for each domain.

  At this time, only the VXLAN ID for wired meshing is derived from the domain seed.

* .. code-block:: lua

    mesh = {
      vxlan = true, -- or false
      -- ...
    },

  In single domain setups, the new *mesh.vxlan* option is mandatory. It should be set to *true* in new
  meshes; existing setups should set it to *false* to retain compatibility with older versions of Gluon.

  In multidomain setups, *mesh.vxlan* defaults to *true* and does not need to be set explicitly.
  It can still be set to *false* for individual domains that should allow wired meshing with existing
  setups, which is also useful for migrating an existing mesh to a multidomain-capable firmware.

* Password change form

  The password change form in the "Advanced settings" is not shown by default anymore, as SSH keys are
  the recommended means of authentication. It is still possible to set a password via SSH while in
  config mode.

  Set

  .. code-block:: lua

    config_mode = {
      remote_login = {
        show_password_form = true,
        -- ...
      },
      -- ...
    },

  to restore the old behaviour.

  When shown, the password form requires a minimum password length of 12 characters now. This requirement
  can be modified using the *config_mode.remote_login.min_password_length* setting.

* Next-node hostnames

  The builtin DNS resolver of Gluon can be configured to resolve a next-node hostname to the
  next-node IP address without querying an upstream DNS server. Since Gluon v2018.1, multiple
  names can be specified. Old configurations setting *next_node.name* to a string must be
  updated to provide an array of strings instead:

  .. code-block:: lua

    next_node = {
      name = { 'nextnode.location.community.example.org' },
      -- ...
    },

i18n
====

It is now possible to override a few labels and descriptions in the configuration
wizard. The available message IDs are listed in :ref:`site-config-mode-texts`.

These new i18n strings are optional; leaving them empty or unset will retain the
default texts.

Internals
*********

Status page rewrite
===================

The status page has been rewritten to simplify the code and reduce its size. Rather than
having a static frontend and retrieving all information via JavaScript, all static information
in the status page is now generated on the node, and JavaScript is only used for dynamic data.

Many status page API endpoints have been removed; for all remaining endpoints, CORS
(Cross-Origin Resource Sharing) has been disabled, as it led to a privacy issues:
malicious websites could access the API via cross-site scripting, determining which
node a user was connected to.

The removal of CORS breaks compatibility with the node switching feature of the
old status page implementation: In the new status page, switching to another node will
reload the whole status page from the target node, while the old implementation would
only switch to another backend host. While this will facilitate future updates, as
frontend and backend always come from the same node and no stable API needs to be maintained,
it prevents switching from the old status page to nodes running the new version.

To achieve all this, the status page was ported to the gluon-web framework. The new status page
also makes use of Gluon's usual i18n facilities now. In addition, the gluon-web-model
package was split out of the gluon-web core package, as model support is only required
for config mode packages, but not for the new status page.

i18n namespaces
===============

In earlier version of Gluon, all gluon-web (formerly LuCI) packages shared the same i18n namespace,
so independent packages could override each others translations (with an arbitrary translation of
the same string "winning"). This issue has been solved by giving each package its own translation
namespace, which is defined by the *package* directive in a package's controller. It is still
possible to access a different i18n namespace (e.g. gluon-web base or site translations), which is
described in :doc:`../dev/web/i18n`.

Package Makefile cleanup
========================

The Makefiles of the individual Gluon packages have been cleaned up significantly by moving a
lot of boilerplate code to *package/gluon.mk*. The new features of *package/gluon.mk* are
explained in detail in :doc:`../dev/packages`.

Site checker
============

* New JSON/Lua path specification

  The old string-based path specifications in site check scripts (e.g. ``'autoupdater.branch'``)
  have been replaced with arrays (``{'autoupdater', 'branch'}``). This will implicitly ensure that
  *autoupdater* is a table when it exists (simplifying checks for deep structures), and it makes it easier
  to specify paths with variable components (by referencing a variable as an array element).

* Alternatives

  The site check library has gained support for *alternatives*. It is now possible to check
  if a configuration satisfies one of multiple checks:

  .. code-block:: lua

    -- foo can be a boolean or a string!
    alternatives(function()
      need_boolean({'foo'})
    end, function()
      need_string({'foo'})
    end)

  As many branches (functions) as necessary can be passed to a single *alternatives* call, which will succeed when
  at least one of the branches succeeds.

batman-adv multicast optimizations
==================================

After various extra rounds of testing and fixes, the batman-adv (compat 15) multicast optimizations were
reenabled: knowledge about potential multicast listeners is gathered and distributed through the mesh again.

This is the next step towards the addition of the actual multicast distribution optimizations, which are
being prepared in `#1357 <https://github.com/freifunk-gluon/gluon/pull/1357>`_. When finished, the optimizations
will help reduce the remaining Layer-2-specific network overhead, e.g. multicasted ICMPv6 messages.

No behaviour changes are expected yet, as the multicast sender side is still disabled.
Once the majority of the mesh network has been updated to Gluon 2018.1, it can be activated on
dedicated nodes by including `#1357 <https://github.com/freifunk-gluon/gluon/pull/1357>`_ in the firmware
build. Test feedback is very welcome.

.. _releases-v2018.1-known-issues:

Known issues
************

* Default TX power on many Ubiquiti devices is too high, correct offsets are unknown (`#94 <https://github.com/freifunk-gluon/gluon/issues/94>`_)

  Reducing the TX power in the Advanced Settings is recommended.

* The MAC address of the WAN interface is modified even when Mesh-on-WAN is disabled (`#496 <https://github.com/freifunk-gluon/gluon/issues/496>`_)

  This may lead to issues in environments where a fixed MAC address is expected (like VMware when promiscuous mode is disallowed).

* Inconsistent respondd API (`#522 <https://github.com/freifunk-gluon/gluon/issues/522>`_)

  The current API is inconsistent and will be replaced eventually. The old API will still be supported for a while.

* Frequent reboots due to out-of-memory or high load due to memory pressure on weak hardware specially in larger meshes
  (`#1243 <https://github.com/freifunk-gluon/gluon/issues/1243>`_)

  Optimizations in Gluon 2018.1 have significantly improved memory usage.
  There are still known bugs leading to unreasonably high load that we hope to
  solve in future releases.

* Configuration loss on upgrade under certain circumstances
  (`#1496 <https://github.com/freifunk-gluon/gluon/issues/1496>`_)

  The issue can only occur when upgrading from 2018.1 and there are multiple
  mirror entries in *site.conf* (specifically, an early failure for one of the
  mirrors, e.g. during DNS resolution, followed by a successful upgrade from a
  different mirror triggers the issue).

  This is a regression in Gluon 2018.1.

* Next-node ARP issue
  (`#1488 <https://github.com/freifunk-gluon/gluon/issues/1488>`_)

  A routing table issue leads to ARP requests being sent from the next-node IPv4 address, but with
  a node-specific source MAC address. This can make the next-node IPv4 address unreachable.

  This is a regression in Gluon 2018.1.