Requests - HTTP Requests Headers

In the previous chapter, we have seen how to make the request and get the response. This chapter will explore a little more on the header section of the URL. So, we are going to look into the following −

  • Understanding Request Headers
  • Custom Headers
  • Response Headers

Understanding Request Headers

Hit any URL in the browser, inspect it and check in developer tool network tab.

You will get response headers, request headers, payload, etc.

For example, consider the following URL −

https://jsonplaceholder.typicode.com/users

View Source

You can get the header details as follows −

Example

import requests
getdata = requests.get('https://jsonplaceholder.typicode.com/users', 
stream = True)
print(getdata.headers)

Output

E:\prequests>python makeRequest.py
{'Date': 'Sat, 30 Nov 2019 05:15:00 GMT', 'Content-Type': 'application/json; 
charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 
'Set-Cookie': '__cfduid=d2b84ccf43c40e18b95122b0b49f5cf091575090900; expires=Mon, 30-De
c-19 05:15:00 GMT; path=/; domain=.typicode.com; HttpOnly', 'X-Powered-By': 
'Express', 'Vary': 'Origin, Accept-Encoding', 'Access-Control-Allow-Credentials': 't
rue', 'Cache-Control': 'max-age=14400', 'Pragma': 'no-cache', 'Expires': '-1', '
X-Content-Type-Options': 'nosniff', 'Etag': 'W/"160d-1eMSsxeJRfnVLRBmYJSbCiJZ1qQ
"', 'Content-Encoding': 'gzip', 'Via': '1.1 vegur', 'CF-Cache-Status': 'HIT', 
'Age': '2271', 'Expect-CT': 'max-age=604800, 
report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"', 'Server': 'cloudflare', 'CF-RAY': '53da574f
f99fc331-SIN'}

To read any http header you can do so as follows −

getdata.headers["Content-Encoding"] // gzip

Custom Headers

You can also send headers to the URL being called as shown below.

Example

import requests
headers = {'x-user': 'test123'}
getdata = requests.get('https://jsonplaceholder.typicode.com/users', 
headers=headers)

The headers passed has to be string, bytestring, or Unicode format. The behavior of the request will not change as per the custom headers passed.

Response Headers

The response headers look like below when you check the URL in the browser developer tool, network tab −

View Sourcecode

To get the details of the headers from the requests module use. Response.headers are as shown below −

Example

import requests
getdata = requests.get('https://jsonplaceholder.typicode.com/users')
print(getdata.headers)

Output

E:\prequests>python makeRequest.py
{'Date': 'Sat, 30 Nov 2019 06:08:10 GMT', 'Content-Type': 'application/json; 
charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 
'Set-Cookie': '__cfduid=de1158f1a5116f3754c2c353055694e0d1575094090; expires=Mon,
30-Dec-19 06:08:10 GMT; path=/; domain=.typicode.com; HttpOnly', 'X-Powered-By': 
'Express', 'Vary': 'Origin, Accept-Encoding', 'Access-Control-Allow-Credentials': 't
rue', 'Cache-Control': 'max-age=14400', 'Pragma': 'no-cache', 'Expires': '-1', '
X-Content-Type-Options': 'nosniff', 'Etag': 'W/"160d-1eMSsxeJRfnVLRBmYJSbCiJZ1qQ
"', 'Content-Encoding': 'gzip', 'Via': '1.1 vegur', 'CF-Cache-Status': 'HIT', 
'Age': '5461', 'Expect-CT': 'max-age=604800, report-uri="https://report-uri.cloudf
lare.com/cdn-cgi/beacon/expect-ct"', 'Server': 'cloudflare', 'CF-RAY': '53daa52f
3b7ec395-SIN'}

You can get any specific header you want as follows −

print(getdata.headers["Expect-CT"])

Output

max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/exp
ect-ct

You can also get the header details by using the get() method.

print(getdata.headers.get("Expect-CT"))

Output

max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/exp
ect-ct
Advertisements