Serving web content directly from a ZIP file

The composability of the Go standard library enables some interesting scenarios for serving static assets on the web.

Go 1.16 introduced the concept of a filesystem interface, which opened some interesting options for code composition. The standard library “archive/zip” package got a Reader.Open method, compatible with this interface. In the same release, the “net/http” package got a new function to plug filesystem implementations into its built-in file server.

Such changes made it trivial to serve static content over HTTP directly from ZIP files, like this:

func run(filename string) error {
	if filename == "" {
		return errors.New("need a path to a zip file")
	}
	zr, err := zip.OpenReader(filename)
	if err != nil {
		return err
	}
	defer zr.Close()
	const addr = "localhost:8000"
	log.Printf("see http://%s/", addr)
	return http.ListenAndServe(addr, http.FileServer(http.FS(zr)))
}

This code transparently serves files from within the ZIP file as if you'd manually unpacked the archive and put its contents to disk.

Browsers almost always announce support for transparent content compression to save bandwidth by sending a special request header: Accept-Encoding: gzip, deflate, br. Its content may vary, but gzip and deflate methods have been supported for decades. It is common to introduce an additional middleware to apply on-the-fly compression or offload this task to CDNs.

Even though this works well, one thing still bothered me a little. One of the most commonly used compression algorithms in ZIP format is deflate — the same compression method almost every browser supports. In the case of serving static assets directly from a ZIP archive, the Go HTTP server first transparently decompresses the content, only to let that content potentially be re-compressed on the following steps of the request processing pipeline.

When I found that Go 1.17 release added support for the OpenRaw method to provide access to the file's content without decompression, I decided to see if I could make it work in the context of the “net/http” package.

Here's my first approach to this:

func Handler(z *zip.Reader) http.Handler {
	// deflate-compressed files, name to index in z.File
	m := make(map[string]int)
	srv := http.FileServer(http.FS(z)) // old logic fallback
	for i := range z.File {
		if z.File[i].Method != zip.Deflate {
			continue
		}
		m[z.File[i].Name] = i
	}
	if len(m) == 0 {
		return srv // no suitable files, return FileServer as is
	}

	// wrapper handler that relies on zip.File.OpenRaw
	// to directly access deflate-compressed content, and
	// falls back to the transparently decompressing old logic
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Add("Vary", "Accept-Encoding")
		if strings.Contains(r.Header.Get("Accept-Encoding"), "deflate") {
			// names inside zip catalog should not start with /
			key := strings.TrimPrefix(r.URL.Path, "/")
			if key == "" {
				key = "index.html"
			}
			if i, ok := m[key]; ok {
				if rd, err := z.File[i].OpenRaw(); err == nil {
					w.Header().Set("Content-Encoding", "deflate")
					io.Copy(w, rd) // sending raw compressed data
					return
				}
			}
		}
		srv.ServeHTTP(w, r) // fallback
	})
}

To my delight, this code works well, just like I hoped it would. The initial example only needs a one-line change to use the new logic:

func run(filename string) error {
	if filename == "" {
		return errors.New("need a path to a zip file")
	}
	zr, err := zip.OpenReader(filename)
	if err != nil {
		return err
	}
	defer zr.Close()
	const addr = "localhost:8000"
	log.Printf("see http://%s/", addr)
	return http.ListenAndServe(addr, Handler(&zr.Reader)) // calling new code
}

This trick is now yours!

The final code got a bit longer than my initial attempt shown here. If you want the code in a reusable form, you can get it as a package:

go get artyom.dev/zipserver@latest

And then use zipserver.Handler in your code.