Menu Developer Moovweb University

Develop Locally: Automatic Request Modification

When a user makes a request to Moovweb (e.g., m.example.com), Moovweb makes a few modifications to that request and then forwards it to the upstream server (e.g., www.example.com). Most of these changes are automatic — this document lists those changes.


Example Request

Here’s an example of an incoming request to Moovweb, together with the outgoing request that the server would issue upon receiving that request.

The HTTP client sends this request to Moovweb:

GET /products/submarine?x-moov-test=false HTTP/1.1
Host: m.example.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16
Referer: http://m.example.com/expensive_gifts
Origin: http://m.example.com
Accept-Encoding: gzip

The server sends this request to the proxied server:

GET /products/submarine HTTP/1.0
Host: example.com
Connection: close
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.133 Safari/534.16
Referer: http://example.com/expensive_gifts
Origin: http://example.com
Accept-Encoding: gzip,deflate
X-Forwarded-For: 100.1.1.1
Via: 1.0 moovweb

Changes

Each section lists one of the ways in which the request is modified. Most of the changes are modifications to the request header, but the first two changes listed are modifications to the request method line. The following headers are modified or added. All other headers are left as-is. Note in particular that the User-Agent header is not modified.

HTTP Version

The HTTP version of the upstream request will be set to 1.0, regardless of whether it was 1.0 or 1.1 in the incoming request. Example status line:

GET / HTTP/1.0

“x-moov-“ Query Parameters

Any query parameter in the path beginning with “x-moov-“ (case insensitive) will be removed. As of writing, the only use for this is to ensure that an asset is not cached by the browser (while not sending any additional parameters to the upstream server). For example, if the incoming request path is:

/potato-pancake?x-moov-cache-killer=1349873&tasty=yes

then the upstream request path will be:

/potato-pancake?tasty=yes

Host

The Host header is rewritten according to the rewriting rules for your project. Usually, but not always, this simply means that the “m.” is removed. For example, if the HTTP client sends:

Host: m.example.com

the server sends:

Host: example.com

Via

Moovweb adds itself to the Via header as required by this section of HTTP. If no Via header exists, one is added. If no Via header exists, the server will send this:

Via: 1.0 moovweb

If the HTTP client sends a Via header, “, 1.0 moovweb” will be appended to the end. For example, if the HTTP client sends:

Via: 1.0 ricky, 1.1 mertz

the server sends:

Via: 1.0 ricky, 1.1 mertz, 1.0 moovweb

X-Forwarded-For

The IP address of the HTTP client will be provided by the de facto X-Forwarded-For header. If the HTTP client sends an X-Forwarded-For header, then the server will add the HTTP client’s IP to the header. If no X-Forwarded-For header exists, one is added.

For example, assume the HTTP client has IP 100.1.1.1. If the HTTP client sends no X-Forwarded-For, the server will send:

X-Forwarded-For: 100.1.1.1

If the HTTP client sends this:

X-Forwarded-For: 200.2.2.2, 300.3.3.3

the server will send this:

X-Forwarded-For: 200.2.2.2, 300.3.3.3, 100.1.1.1

Referer and Origin

NOTE: The word “Referrer” is intentionally misspelled throughout this document to reflect the spelling of the HTTP Referer header.

The server attempts to mimic a normal user of the website, so the Referer and Origin headers are rewritten to reflect this. The rewriting happens based on the project’s rewriter rules. Usually this simply means removing an “m.”. For example, if the HTTP client sends:

Referer: http://m.example.com/snakes_on_planes
Origin: http://m.example.com

the server sends:

Referer: http://example.com/snakes_on_planes
Origin: http://example.com

Accept-Encoding

The server supports deflate and gzip encoding, so the Accept-Encoding header will always reflect this. If the HTTP client sends an Accept-Encoding header, it will be modified to look like this:

Accept-Encoding: gzip,deflate

NOTE: If the HTTP client does not send an Accept-Encoding header, the server will not add one.

Connection

The server does not support the Connection: keep-alive header. If one is encountered, it will be changed to Connection: close.

Body

Any request body (as often contained in a POST) will be forwarded unmodified.


FAQ

Is the User-Agent header modified?

Like all other headers not mentioned in the section on headers, the User-Agent header is not modified. The proxied server will see the user agent of the HTTP client.

However, the server does have a configuration option that allows it to send a particular value for the user agent. If the server is configured in this way it will send a constant user agent instead of forwarding the customer’s user agent.