Why Mix no longer installs from HTTP(S) URLs

Posted 2020-01-01 20:02:12.309037

As of Elixir 1.9.3 the use of HTTP or HTTPS URLs in various Mix commands is deprecated, and in 1.10 this functionality will be disabled altogether. This affects the archive.install, escript.install and local.rebar Mix commands.

In this post I will explain why the use of URLs was a security risk, what are the alternatives, and why those alternatives (including Hex) are safe.

Why?

The issue here is that neither Erlang/OTP nor Elixir ships with a CA trust store, and picking up the CA trust store from the operating system is quite hard to do in a reliable, cross-platform way. Without a trust store it is not possible to verify the server’s certificate, which means there is no protection against man-in-the-middle (MitM) attacks. In case of plain HTTP URLs this must have been pretty obvious to users, but when using HTTPS people might be forgiven for thinking they’d be safe from such attacks.

The affected Mix tasks were used to fetch code for local execution or for inclusion into a software product that might be used in production. A MitM attack would allow the attacker to inject malicious code, stealing data or taking over the developer’s machine, build server or even production systems.

Alternatives

So I think you’ll agree that the removal of this dangerous functionality is a good thing. But what are the alternatives?

Hex

Fetching archives or escripts from hex, e.g. using mix archive.install hex package, uses the Hex client to fetch the files from the Hex CDN. The Hex client includes a CA trust store, and it uses the appropriate TLS configuration to make the :httpc client correctly verify the server’s certificate.

If you are wondering if that’s not a catch-22, since the Hex client itself is installed by Mix, keep reading…

Git / GitHub

When the necessary files are not hosted on Hex, but are accessible through Git, Mix can delegate the fetching of the files to the Git client. This obviously requires a Git client to be installed. The Git client does its own server certificate verification, independent of Mix.

External HTTP client

As a last resort, users can use any trusted HTTP client to fetch the file from an HTTPS URL, and then pass a local path to the Mix command. This works not just for archive and escript installation, but also for the local.rebar command. The tools recommended in the deprecation message (curl, wget and Windows powershell) all use the OS CA trust store or bring their own.

It is a little less convenient, since Mix does not invoke the HTTP client: the user must perform this extra step manually prior to running Mix. This should not be an issue for scripted installations, a common use-case for URL-based installs.

So what about mix local.hex?

So far we have ignored an important Mix command that also fetches code from a server: mix local.hex. Isn’t this command also affected by the lack of a CA certificate? And how about mix local.rebar (without additional arguments)?

The answer is that Mix performs a checksum (more accurately: SHA-512 digest) verification on the downloaded Hex and Rebar clients. In fact, the option to verify the downloaded data against a checksum was also available for other URL-based downloads, but it was rarely used outside of these built-in tasks.

Where does the checksum come from? From CSV files, one for Hex, one for Rebar and one for Rebar3, fetched from the Hex CDN over HTTPS.

Wait a minute… we’re back to square one: how can we rely on checksums from a file that was fetched over an insecure channel? The answer is: signatures. If you append ‘.signed’ to any of the above CSV URLs you’ll get a digital signature. Mix ships with a public key that is used to verify these signatures before using the information contained in the corresponding file. Run mix help local.public_keys for more information.

So there you have it: the public key is the trust anchor that allows the Hex client to be installed over an otherwise insecure HTTPS connection.

For more information about how Hex ensures the integrity of packages it installs, check out this post.