H2o and HTTP/2

The HTTP/21 RFC (7540)2 is now 4 months old (May, 2015).

Its time for testing. Initially, I was using nginx3. I’ve used it for a while and been happy with it. I’d gone through a migration from wordpress to pelican with it, and even with PHP, it was pleasantly lightweight and responsive. Now that I’m using pelican, I don’t have any fancy requirements for this site, its all static content. No PHP, Ruby, or even CGI. After getting the new version of nginx deployed (1.9.x + HTTP/2 patches), things appeared to be working, until I looked at the logs. The logs claimed everything was being served as HTTP/1.1. After a little debugging, it seems that the logging was incorrect. It was serving HTTP/2, but logging it as 1.1.


I really wanted the logs to be acurate (if not, what is the purpose of logs? Yes, I know. Development patches for nginx. They’ll get things working before it lands in production. Fine. I want something now). After a little searching, I found h2o4. h2o is a fast and secure HTTP/2 server written in C. h2o can be used as an insecure HTTP server and/or a secure HTTPS (TLS5) server supporting HTTP 1.0, 1.1 and the shiny new HTTP/2 standard. Both h2o and nginx are very lightweight, and very fast, but h2o has an improved parser for html and thus comes out a little ahead. Oh yeah, h2o also reports things properly in the logs.

Goals of HTTP/2

Negotiation mechanism that allows clients and servers to elect to use HTTP 1.1, 2.0, or potentially other non-HTTP protocols.

Maintain high-level compatibility with HTTP 1.1 (for example with methods, status codes, and URIs, and most header fields)

Decrease latency to improve page load speed in web browsers by considering:

  • Data compression of HTTP headers
  • Server push technologies
  • Fixing the head-of-line blocking problem in HTTP 1
  • Loading page elements in parallel over a single TCP connection

Support common existing use cases of HTTP, such as desktop web browsers, mobile web browsers, web APIs, web servers at various scales, proxy servers, reverse proxy servers, firewalls, and content delivery networks


Differences between HTTP/1.x and HTTP/2

The changes for HTTP/2 don’t require changes for existing web applications to
continue working, however new applications are able to take advantage of the new features for additional speed.

HTTP/2 leaves most of HTTP 1.1’s high level syntax, things like methods, status
codes, header fields, and URIs, unchanged. What was modified is how the data is
framed and transported from the server to the client.

Websites that are efficient work to minimize the number of client requests
required to render an entire page. One example is minifying (reducing the size
of the code files and combining multiple code files into a single file, all without reducing its ability to function) resources such as stylesheets and javascript.

HTTP/2 allows the server to “push” content. The server can respond with
additional data which the client hasn’t requested yet, but will need (for example, the server could push images, or style sheets before the browser has had time to parse the DOM and discover it needs to request them. This also eliminates an additional request cycle from the browser.

Additional performance improvements in HTTP/2 come from the multiplexing of requests and responses which avoids the head-of-line blocking problem in HTTP 1 (this is present, even when HTTP pipelining is used), header compression (HPACK7), and prioritization of requests. Header compression will make a significant different to the mobile arena, where headers can be a significant percentage of the total data transmitted to the client.

Browser Compatibility

The following browsers are compatible with HTTP/2:

Browser Version
Chrome 40 or newer (Currently only HTTP/2 over TLS is implemented)
Chrome for iOS
Firefox 36 or newer (Currently only HTTP/2 over TLS is implemented)
Internet Explorer 11 or newer (Currently only HTTP/2 over TLS is implemented8. Only on Windows 10)
Microsoft Edge
Safari 9 or newer

Installing h2o

This is written for FreeBSD, if you choose to use something else, you will need to adjust the process for installation accordingly.

Installing h2o is very simple. You can build it from source (ports), or as a pkg. I’ll include both examples:

Source (I’ve stripped lines with warnings):

  1[root@www h2o 191 ]$ make install clean
  2===>  License MIT accepted by the user
  3===>   h2o-1.4.4_1 depends on file: /usr/local/sbin/pkg - found
  4===> Fetching all distfiles required by h2o-1.4.4_1 for building
  5===>  Extracting for h2o-1.4.4_1
  6=> SHA256 Checksum OK for h2o-h2o-v1.4.4_GH0.tar.gz.
  7===>  Patching for h2o-1.4.4_1
  8===>  Applying FreeBSD patches for h2o-1.4.4_1
  9===>   h2o-1.4.4_1 depends on file: /usr/local/bin/cmake - found
 10===>  Configuring for h2o-1.4.4_1
 11===>  Performing in-source build
 12/bin/mkdir -p /usr/ports/www/h2o/work/h2o-1.4.4
 13-- The C compiler identification is Clang 3.4.1
 14-- Check for working C compiler: /usr/bin/cc
 15-- Check for working C compiler: /usr/bin/cc -- works
 16-- Detecting C compiler ABI info
 17-- Detecting C compiler ABI info - done
 18-- Detecting C compile features
 19-- Detecting C compile features - done
 20-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.28")
 21-- Looking for include file pthread.h
 22-- Looking for include file pthread.h - found
 23-- Found Threads: TRUE
 24-- Found OpenSSL: /usr/local/lib/libssl.so;/usr/local/lib/libcrypto.so (found version "1.0.2d")
 25-- checking for module 'libuv>=1.0.0'
 26--   package 'libuv>=1.0.0' not found
 28-- checking for module 'libwslay'
 29--   package 'libwslay' not found
 31-- checking for module 'libmruby'
 32--   package 'libmruby' not found
 34-- Configuring done
 35-- Generating done
 36CMake Warning:
 37  Manually-specified variables were not used by the project:
 48-- Build files have been written to: /usr/ports/www/h2o/work/h2o-1.4.4
 49===>  Building for h2o-1.4.4_1
 50Scanning dependencies of target h2o
 51[  2%] Building C object CMakeFiles/h2o.dir/deps/cloexec/cloexec.c.o
 52[  2%] Building C object CMakeFiles/h2o.dir/deps/libyrmcds/close.c.o
 53[  2%] Building C object CMakeFiles/h2o.dir/deps/libyrmcds/connect.c.o
 54[  5%] Building C object CMakeFiles/h2o.dir/deps/libyrmcds/recv.c.o
 55[  5%] Building C object CMakeFiles/h2o.dir/deps/libyrmcds/send.c.o
 56[  5%] Building C object CMakeFiles/h2o.dir/deps/libyrmcds/socket.c.o
 57[  7%] Building C object CMakeFiles/h2o.dir/deps/libyrmcds/strerror.c.o
 58[  7%] Building C object CMakeFiles/h2o.dir/deps/picohttpparser/picohttpparser.c.o
 59[  7%] Building C object CMakeFiles/h2o.dir/lib/common/file.c.o
 60[ 10%] Building C object CMakeFiles/h2o.dir/lib/common/hostinfo.c.o
 61[ 10%] Building C object CMakeFiles/h2o.dir/lib/common/http1client.c.o
 62[ 10%] Building C object CMakeFiles/h2o.dir/lib/common/memcached.c.o
 63[ 10%] Building C object CMakeFiles/h2o.dir/lib/common/memory.c.o
 64[ 12%] Building C object CMakeFiles/h2o.dir/lib/common/multithread.c.o
 65[ 12%] Building C object CMakeFiles/h2o.dir/lib/common/serverutil.c.o
 66[ 12%] Building C object CMakeFiles/h2o.dir/lib/common/socket.c.o
 67[ 15%] Building C object CMakeFiles/h2o.dir/lib/common/socketpool.c.o
 68[ 15%] Building C object CMakeFiles/h2o.dir/lib/common/string.c.o
 69[ 15%] Building C object CMakeFiles/h2o.dir/lib/common/time.c.o
 70[ 17%] Building C object CMakeFiles/h2o.dir/lib/common/timeout.c.o
 71[ 17%] Building C object CMakeFiles/h2o.dir/lib/common/url.c.o
 72[ 17%] Building C object CMakeFiles/h2o.dir/lib/core/config.c.o
 73[ 20%] Building C object CMakeFiles/h2o.dir/lib/core/configurator.c.o
 74[ 20%] Building C object CMakeFiles/h2o.dir/lib/core/context.c.o
 75[ 20%] Building C object CMakeFiles/h2o.dir/lib/core/headers.c.o
 76[ 23%] Building C object CMakeFiles/h2o.dir/lib/core/proxy.c.o
 77[ 23%] Building C object CMakeFiles/h2o.dir/lib/core/request.c.o
 78[ 23%] Building C object CMakeFiles/h2o.dir/lib/core/token.c.o
 79[ 25%] Building C object CMakeFiles/h2o.dir/lib/core/util.c.o
 80[ 25%] Building C object CMakeFiles/h2o.dir/lib/handler/access_log.c.o
 81[ 25%] Building C object CMakeFiles/h2o.dir/lib/handler/chunked.c.o
 82[ 28%] Building C object CMakeFiles/h2o.dir/lib/handler/expires.c.o
 83[ 28%] Building C object CMakeFiles/h2o.dir/lib/handler/fastcgi.c.o
 84[ 28%] Building C object CMakeFiles/h2o.dir/lib/handler/file.c.o
 85[ 28%] Building C object CMakeFiles/h2o.dir/lib/handler/headers.c.o
 86[ 30%] Building C object CMakeFiles/h2o.dir/lib/handler/mimemap.c.o
 87[ 30%] Building C object CMakeFiles/h2o.dir/lib/handler/proxy.c.o
 88[ 30%] Building C object CMakeFiles/h2o.dir/lib/handler/redirect.c.o
 89[ 33%] Building C object CMakeFiles/h2o.dir/lib/handler/reproxy.c.o
 90[ 33%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/access_log.c.o
 91[ 33%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/expires.c.o
 92[ 35%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/fastcgi.c.o
 93[ 35%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/file.c.o
 94[ 35%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/headers.c.o
 95[ 38%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/proxy.c.o
 96[ 38%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/redirect.c.o
 97[ 38%] Building C object CMakeFiles/h2o.dir/lib/handler/configurator/reproxy.c.o
 98[ 41%] Building C object CMakeFiles/h2o.dir/lib/http1.c.o
 99[ 41%] Building C object CMakeFiles/h2o.dir/lib/http2/connection.c.o
100[ 41%] Building C object CMakeFiles/h2o.dir/lib/http2/frame.c.o
101[ 43%] Building C object CMakeFiles/h2o.dir/lib/http2/hpack.c.o
102[ 43%] Building C object CMakeFiles/h2o.dir/lib/http2/scheduler.c.o
103[ 43%] Building C object CMakeFiles/h2o.dir/lib/http2/stream.c.o
104[ 46%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/api.c.o
105[ 46%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/dumper.c.o
106[ 46%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/emitter.c.o
107[ 46%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/loader.c.o
108[ 48%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/parser.c.o
109[ 48%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/reader.c.o
110[ 48%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/scanner.c.o
111[ 51%] Building C object CMakeFiles/h2o.dir/deps/yaml/src/writer.c.o
112[ 51%] Building C object CMakeFiles/h2o.dir/src/ssl.c.o
113[ 51%] Building C object CMakeFiles/h2o.dir/src/main.c.o
114[ 53%] Linking C executable h2o
115[ 53%] Built target h2o
116Scanning dependencies of target libh2o-evloop
117[ 53%] Building C object CMakeFiles/libh2o-evloop.dir/deps/cloexec/cloexec.c.o
118[ 53%] Building C object CMakeFiles/libh2o-evloop.dir/deps/libyrmcds/close.c.o
119[ 56%] Building C object CMakeFiles/libh2o-evloop.dir/deps/libyrmcds/connect.c.o
120[ 56%] Building C object CMakeFiles/libh2o-evloop.dir/deps/libyrmcds/recv.c.o
121[ 56%] Building C object CMakeFiles/libh2o-evloop.dir/deps/libyrmcds/send.c.o
122[ 58%] Building C object CMakeFiles/libh2o-evloop.dir/deps/libyrmcds/socket.c.o
123[ 58%] Building C object CMakeFiles/libh2o-evloop.dir/deps/libyrmcds/strerror.c.o
124[ 58%] Building C object CMakeFiles/libh2o-evloop.dir/deps/picohttpparser/picohttpparser.c.o
125[ 61%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/file.c.o
126[ 61%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/hostinfo.c.o
127[ 61%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/http1client.c.o
128[ 64%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/memcached.c.o
129[ 64%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/memory.c.o
130[ 64%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/multithread.c.o
131[ 66%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/serverutil.c.o
132[ 66%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/socket.c.o
133[ 66%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/socketpool.c.o
134[ 69%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/string.c.o
135[ 69%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/time.c.o
136[ 69%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/timeout.c.o
137[ 69%] Building C object CMakeFiles/libh2o-evloop.dir/lib/common/url.c.o
138[ 71%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/config.c.o
139[ 71%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/configurator.c.o
140[ 71%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/context.c.o
141[ 74%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/headers.c.o
142[ 74%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/proxy.c.o
143[ 74%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/request.c.o
144[ 76%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/token.c.o
145[ 76%] Building C object CMakeFiles/libh2o-evloop.dir/lib/core/util.c.o
146[ 76%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/access_log.c.o
147[ 79%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/chunked.c.o
148[ 79%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/expires.c.o
149[ 79%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/fastcgi.c.o
150[ 82%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/file.c.o
151[ 82%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/headers.c.o
152[ 82%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/mimemap.c.o
153[ 84%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/proxy.c.o
154[ 84%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/redirect.c.o
155[ 84%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/reproxy.c.o
156[ 87%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/access_log.c.o
157[ 87%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/expires.c.o
158[ 87%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/fastcgi.c.o
159[ 87%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/file.c.o
160[ 89%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/headers.c.o
161[ 89%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/proxy.c.o
162[ 89%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/redirect.c.o
163[ 92%] Building C object CMakeFiles/libh2o-evloop.dir/lib/handler/configurator/reproxy.c.o
164[ 92%] Building C object CMakeFiles/libh2o-evloop.dir/lib/http1.c.o
165[ 92%] Building C object CMakeFiles/libh2o-evloop.dir/lib/http2/connection.c.o
166[ 94%] Building C object CMakeFiles/libh2o-evloop.dir/lib/http2/frame.c.o
167[ 94%] Building C object CMakeFiles/libh2o-evloop.dir/lib/http2/hpack.c.o
168[ 94%] Building C object CMakeFiles/libh2o-evloop.dir/lib/http2/scheduler.c.o
169[ 97%] Building C object CMakeFiles/libh2o-evloop.dir/lib/http2/stream.c.o
170[ 97%] Linking C static library libh2o-evloop.a
171[ 97%] Built target libh2o-evloop
172Scanning dependencies of target setuidgid
173[ 97%] Building C object CMakeFiles/setuidgid.dir/src/setuidgid.c.o
174[100%] Linking C executable setuidgid
175[100%] Built target setuidgid
176===>  Staging for h2o-1.4.4_1
177===>   Generating temporary packing list
178[ 53%] Built target h2o
179[ 97%] Built target libh2o-evloop
180[100%] Built target setuidgid
181Installing the project stripped...
182-- Install configuration: "Release"
183-- Installing: /usr/ports/www/h2o/work/stage/usr/local/bin/h2o
184-- Set runtime path of "/usr/ports/www/h2o/work/stage/usr/local/bin/h2o" to "/usr/local/lib"
185-- Installing: /usr/ports/www/h2o/work/stage/usr/local/share/h2o/setuidgid
186-- Up-to-date: /usr/ports/www/h2o/work/stage/usr/local/include
187-- Installing: /usr/ports/www/h2o/work/stage/usr/local/share/h2o/annotate-backtrace-symbols
188-- Installing: /usr/ports/www/h2o/work/stage/usr/local/share/h2o/fetch-ocsp-response
189-- Installing: /usr/ports/www/h2o/work/stage/usr/local/share/h2o/kill-on-close
190-- Installing: /usr/ports/www/h2o/work/stage/usr/local/share/h2o/start_server
191/bin/mkdir -p /usr/ports/www/h2o/work/stage/usr/local/share/doc/h2o  /usr/ports/www/h2o/work/stage/usr/local/etc/h2o  /usr/ports/www/h2o/work/stage/var/log/h2o/
192install  -m 0644 /usr/ports/www/h2o/work/h2o-1.4.4/README.md /usr/ports/www/h2o/work/stage/usr/local/share/doc/h2o
193install  -m 0644  /usr/ports/www/h2o/files/h2o.conf.sample  /usr/ports/www/h2o/work/stage/usr/local/etc/h2o/h2o.conf.sample
194====> Compressing man pages (compress-man)
195===> Staging rc.d startup script(s)
196===>  Installing for h2o-1.4.4_1
197===>   Registering installation for h2o-1.4.4_1
198Installing h2o-1.4.4_1...
200      This port has installed the following files which may act as network
201      servers and may therefore pose a remote security risk to the system.
204      This port has installed the following startup scripts which may cause
205      these network services to be started at boot time.
208      If there are vulnerabilities in these programs there may be a security
209      risk to the system. FreeBSD makes no guarantee about the security of
210      ports included in the Ports Collection. Please type 'make deinstall'
211      to deinstall the port if this is a concern.
213      For more information, and contact details about the security
214      status of this software, see the following webpage:
216===>  Cleaning for h2o-1.4.4_1
218                        Time spent in user mode   (CPU seconds) : 24.929s
219                        Time spent in kernel mode (CPU seconds) : 8.296s
220                        Total time                              : 0:33.89s
221                        CPU utilisation (percentage)            : 97.9%
222                        Times the process was swapped           : 0
223                        Times of major page faults              : 56
224                        Times of minor page faults              : 502856
225[root@www h2o 192 ]$


1pkg install www/h2o

Next, we need a bit in the /etc/rc.conf to allow us to start the h2o service:


Configuring h2o

I’m going to show the config file for my test/development machine. Its similar to production, and has been adapted from Calomel9. There are other example config files available here .

h2o has a straight forward config file. Many of the directives (in key/value form) fall under the individual host section.

 1# Original h2o config file from https://calomel.org/h2o.html
 3# global configuration
 4access-log: /var/log/h2o/h2o_access.log
 5error-log: /var/log/h2o/h2o_error.log
 6expires: off
 7file.dirlisting: off
 8file.send-gzip: on
 9limit-request-body: 1024
10pid-file: /var/run/h2o.pid
12# mime types, additional types and redefine of *.html
14  application/atom+xml: .xml
15  application/zip: .zip
16  "text/html; charset=utf-8": .html
18# ssl resume, cache and ticket methods supported, stored in RAM 
20  mode: all
22# listening ports and protocols 
24  "test.cryptomonkeys.com:8000":
25    listen:
26      host:
27      port: 8000
28    paths:
29      /:
30        redirect:
31          status: 301
32          url: https://test.cryptomonkeys.com:8443/
34  "test.cryptomonkeys.com:8443":
35    listen:
36      host:
37      port: 8443
38      ssl:
39        certificate-file: /usr/local/etc/sslmate/test.cryptomonkeys.com.crt
40        key-file: /usr/local/etc/sslmate/test.cryptomonkeys.com.key
41        cipher-preference: server
43        minimum-version: TLSv1.2
44  paths:
45    /:
46      file.dir: /usr/local/www/cryptomonkeys.com
47    /stats:
48      file.dir: /usr/local/www/cryptomonkeys.com_stats
50### EOF ###

Once the bits are placed in the config file (/usr/local/etc/h2o/h2o.conf), we can start the service with:

1sudo service h2o start

You should now be able to point a browser at your h2o server and load a page. If you are successful, you should see some log bits similar to what is in the next section.

h2o Logs

The location of the logs is defined in the h2o.conf file. Mine are in /var/log/h2o, and I give each domain its own log file. For example: /var/log/h2o/cryptomonkeys.com.log

There are a couple ways you can check from the browser whether you are communicating over HTTP/2. If you have an HTTP/2 capable browser (I’ve been using Firefox, because it has a handy plugin10 that will show me whether I’m connecting with HTTP/2 in the URL bar. It also shows me SPDY11 connections, but I’m less interested in that since SPDY is being phased out for HTTP/2).

You can also use a plugin like Firebug12 to inspect the communication between the browser and the server.

Lastly, when looking at your server logs (probably hidden away under /var/log/h2o/*.log) you should see a note that includes the magic bits HTTP/2.

1127.0.0.1 - - [08/Sep/2015:23:46:18 +0000] "GET / HTTP/2" 200 3169 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:41.0) Gecko/20100101 Firefox/41.0"

Troubleshooting and Debugging

If you need to do debugging of HTTP/2, and tools like firebug aren’t sufficient, you can take a look at nghttp213. They have implemented a client, server, and proxy that all speak HTTP/2. Also of note, Curl14 can be compiled with HTTP/2 support.

Footnotes and References