HttpClientはデフォルトでAccept Headerを付けない

  • このエントリーをはてなブックマークに追加

ApacheのHttpClinetはデフォルトでAcceptヘッダを付与しません。
例えば、簡単なサンプルを次に示します。

SSLContextConnectionManager,RequestConfigを設定していませんが、そこは本質的ではないので省略しています。

1
2
3
4
5
6
7
8
9
10
HttpClientBuilder clientBuilder = HttpClientBuilder.create();
try (CloseableHttpClient client = clientBuilder.build()) {
HttpGet getMethod = new HttpGet();
getMethod.setURI(URI.create("https://httpstat.us/200"));
try (CloseableHttpResponse resp = client.execute(getMethod)) {
String body = EntityUtils.toString(resp.getEntity(), StandardCharsets.UTF_8);
logger.info("header:{}", resp.getStatusLine());
logger.info("body:{}", body);
}
}

HttpClientのDebugLogをみたいので、LogレベルをDEBUGに設定していまいます。あとコピペしやすいようにPatternをちょっとだけ修正。

1
2
3
4
5
6
7
8
9
10
Logger rootLogger = (Logger) LoggerFactory.getLogger(Logger.ROOT_LOGGER_NAME);
rootLogger.setLevel(Level.DEBUG);

PatternLayoutEncoder ple = new PatternLayoutEncoder();
ple.setContext(rootLogger.getLoggerContext());
ple.setPattern("%-50class{1} %msg%n");
ple.start();

ConsoleAppender consoleAppender = (ConsoleAppender) rootLogger.getAppender("console");
consoleAppender.setEncoder(ple);

実行結果です。必要なところだけを抜き出しています。

RunningLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
o.a.h.i.e.MainClientExec                           Executing request GET /200 HTTP/1.1
o.a.h.i.e.MainClientExec Target auth state: UNCHALLENGED
o.a.h.i.e.MainClientExec Proxy auth state: UNCHALLENGED

o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 >> GET /200 HTTP/1.1
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 >> Host: httpstat.us
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 >> Connection: Keep-Alive
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.4 (Java/11.0.2)
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 >> Accept-Encoding: gzip,deflate
o.a.h.i.c.Wire http-outgoing-0 >> "GET /200 HTTP/1.1[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 >> "Host: httpstat.us[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.4 (Java/11.0.2)[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 >> "[\r][\n]"

o.a.h.i.c.Wire http-outgoing-0 << "HTTP/1.1 200 OK[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "Cache-Control: private[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "Server: Microsoft-IIS/10.0[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "X-AspNetMvc-Version: 5.1[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "Access-Control-Allow-Origin: *[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "X-AspNet-Version: 4.0.30319[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "X-Powered-By: ASP.NET[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "Set-Cookie: ARRAffinity=beec3692495883b8df6195e900c12f49514e054d865a22ad2951c84f51dbaf93;Path=/;HttpOnly;Domain=httpstat.us[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "Date: Sun, 17 Mar 2019 08:21:05 GMT[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "Content-Length: 0[\r][\n]"
o.a.h.i.c.Wire http-outgoing-0 << "[\r][\n]"

o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << HTTP/1.1 200 OK
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << Cache-Control: private
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << Server: Microsoft-IIS/10.0
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << X-AspNetMvc-Version: 5.1
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << Access-Control-Allow-Origin: *
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << X-AspNet-Version: 4.0.30319
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << X-Powered-By: ASP.NET
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << Set-Cookie: ARRAffinity=beec3692495883b8df6195e900c12f49514e054d865a22ad2951c84f51dbaf93;Path=/;HttpOnly;Domain=httpstat.us
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << Date: Sun, 17 Mar 2019 08:21:05 GMT
o.a.h.i.c.LoggingManagedHttpClientConnection http-outgoing-0 << Content-Length: 0
o.a.h.i.e.MainClientExec Connection can be kept alive indefinitely

o.a.HttpClientSample header:HTTP/1.1 200 OK
o.a.HttpClientSample body:

これを見るとAcceptヘッダを送信していません。

何が問題か?

ブラウザやcurlはデフォルトでAcceptを送信しており、そこには*/*が含まれています。
参照: https://developer.mozilla.org/ja/docs/Web/HTTP/Content_negotiation/List_of_default_Accept_values

サンプルに利用したhttpstatでAcceptヘッダがある場合と無い場合とで確認してみましょう。

include_accept_header
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ curl http://httpstat.us/200 -v
* Trying 23.99.0.12...
* TCP_NODELAY set
* Connected to httpstat.us (23.99.0.12) port 80 (#0)
> GET /200 HTTP/1.1
> Host: httpstat.us
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: private
< Content-Length: 6
< Content-Type: text/plain; charset=utf-8
< Server: Microsoft-IIS/10.0
< X-AspNetMvc-Version: 5.1
< Access-Control-Allow-Origin: *
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Set-Cookie: ARRAffinity=beec3692495883b8df6195e900c12f49514e054d865a22ad2951c84f51dbaf93;Path=/;HttpOnly;Domain=httpstat.us
< Date: Sun, 17 Mar 2019 08:33:52 GMT
<
* Connection #0 to host httpstat.us left intact
200 OK
exclude_accept_header
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ curl -H "Accept:"  http://httpstat.us/200 -v
* Trying 23.99.0.12...
* TCP_NODELAY set
* Connected to httpstat.us (23.99.0.12) port 80 (#0)
> GET /200 HTTP/1.1
> Host: httpstat.us
> User-Agent: curl/7.58.0
>
< HTTP/1.1 200 OK
< Cache-Control: private
< Server: Microsoft-IIS/10.0
< X-AspNetMvc-Version: 5.1
< Access-Control-Allow-Origin: *
< X-AspNet-Version: 4.0.30319
< X-Powered-By: ASP.NET
< Set-Cookie: ARRAffinity=beec3692495883b8df6195e900c12f49514e054d865a22ad2951c84f51dbaf93;Path=/;HttpOnly;Domain=httpstat.us
< Date: Sun, 17 Mar 2019 08:36:54 GMT
< Content-Length: 0
<
* Connection #0 to host httpstat.us left intact

Accpetヘッダを送信しないと、Content-Lengthヘッダが0になってBODYが空のレスポンスが返ってきました。

このサイトに限らず、例えばAcceptヘッダでapplication/jsonだとJSON形式で返すようなサイトでは、Frameworkによってはこういう挙動になるかもしれません。

どう書けばいいのか

いくつかやり方がありますがDefaultHeaderを設定するのが楽だと思います。

1
2
3
List<BasicHeader> defaultHeaders = Arrays.asList(new BasicHeader("Accept", "*/*"));
HttpClientBuilder clientBuilder = HttpClientBuilder.create()
.setDefaultHeaders(defaultHeaders);

もしくはGetMethodに設定します。

1
getMethod.setHeader("Accept", "*/*");