What are HTTP Headers & Understand different types of HTTP headers

What are HTTP headers?

HTTP headers are additional pieces of information sent along with an HTTP request or response. They provide important metadata about the request or response, such as the content type, caching directives, authentication credentials, and more. HTTP headers consist of a name-value pair and are separated by a colon.

There are several types of HTTP headers, including:

1. HTTP Request Headers

These headers are sent by the client to the server to provide information about the request or the client itself. Some commonly used request headers include:

• Accept: Specifies the MIME(Multipurpose Internet Mail Extensions) types accepted by the client.
• User-Agent: Identifies the client software or user agent making the request.
• Authorization: Contains authentication credentials for accessing protected resources.
• Content-Type: Indicates the media type of the request body.

2. HTTP Response Headers

These headers are sent by the server in response to a client's request and provide information about the server or the response itself. Some commonly used response headers include:

• Content-Type: Specifies the media type of the response body.
• Cache-Control: Controls caching behavior for the response.
• Set-Cookie: Sets a cookie in the client's browser for maintaining the session state.
• Location: Used in redirection to specify the new location for the requested resource.

3. General Headers

These headers are used in both requests and responses and provide general information about the message. Some examples include:

• Date: Specifies the date and time when the message was generated.
• Connection: Controls whether the connection should be kept alive or closed.

4. Entity Headers

These headers provide information about the body or entity of the message. Examples include:

• Content-Length: Indicates the length of the content in bytes.
• Content-Encoding: Specifies the encoding applied to the response body.

HTTP headers allow clients and servers to communicate additional information beyond the basic request or response content, enabling various functionalities and optimizations in web communication.

Check out HTTP headers that every developer should now article to know about important headers.

Categories of different HTTP Headers:

1. Security Headers

These headers help enhance the security of web applications and protect against various types of attacks. These headers provide instructions to the web browser on how to handle and enforce certain security policies. They help mitigate risks associated with common web vulnerabilities, such as cross-site scripting (XSS), cross-site request forgery (CSRF), clickjacking, and more.

For example:

Strict-Transport-Security: max-age=31536000; includeSubDomains 

Content-Security-Policy: **default-src** 'self'; **script-src** 'self' 'unsafe-inline';

Referrer-Policy: strict-origin-when-cross-origin 

• X-Content-Type-Options: Prevents MIME(Multipurpose Internet Mail Extensions) type sniffing.
• Strict-Transport-Security: Enforces the use of HTTPS for secure communication.
• X-XSS-Protection: Enables the browser's cross-site scripting (XSS) filter.
• Content-Security-Policy: Content-Security-Policy defines the allowed sources for web content. this response header is a security mechanism used to mitigate the risks associated with cross-site scripting (XSS) attacks and other types of code injection vulnerabilities in web applications. It allows web developers to define a set of policies that dictate which resources and scripts can be loaded and executed on a web page.
• X-Frame-Options: X-frame-options prevent clickjacking attacks by controlling whether a web page can be displayed in an iframe on another site.
• Referrer-Policy: Determines how much information about the referring URL is included in the referrer header when navigating between different websites.
• Feature-Policy: Controls which web platform features and APIs are allowed to be used on a web page, reducing the potential attack surface.

2. Authentication Headers:

These headers are used for authentication and authorization purposes information during the communication between a client and a server. These headers allow clients to authenticate themselves and include credentials or tokens to access protected resources. Authentication headers play a crucial role in securing API endpoints, web services, and other protected resources. They allow clients to authenticate themselves and gain authorized access to restricted content or perform actions on behalf of authenticated users.

For example:

Authorization: Basic base64(username:password)

WWW-Authenticate: **Basic** realm="Secure Area"

Authorization: Bearer <access_token>

• Authorization: This header is used to send authentication credentials to the server. It typically includes a username and password or a token for authentication. The most common authentication scheme used with this header is Basic authentication, where the credentials are Base64-encoded.
• WWW-Authenticate: This header is sent by the server to request authentication from the client. It specifies the authentication scheme and any additional parameters required for authentication. When the client receives this header, it can respond with the appropriate credentials using the Authorization header.
• Bearer: This header is commonly used for token-based authentication, such as OAuth 2.0 or JSON Web Tokens (JWT). The client includes an access token in the Authorization header using the Bearer scheme.

You can check out Requestly docs to modify headers using Requestly.

3. Caching-related Headers

Cache headers in HTTP are used to control caching behavior for responses, allowing clients and intermediary servers to store and reuse the response instead of requesting it again from the origin server. Cache headers should be set according to the specific caching requirements and business logic of your application. Improper cache configurations can lead to incorrect or stale responses being served to clients.

Example:

*Cache**-Control: **public**, **max**-age=3600

Expires: Wed, 21 Jul 2023 10:00:00 GMT

ETag: "abcd1234"

Last:** Modified: Tue, 13 Jul 2023 15:30:00 GMT

• Cache-Control: Specifies caching directives such as max-age, no-cache, no-store, etc.
• Expires: Indicates the date/time after which the response is considered stale.
• ETag: Provides a unique identifier for a specific version of a resource.

4. CORS Headers

HTTP CORS headers (Cross-Origin Resource Sharing headers) are used to control cross-origin requests, allowing servers to specify which origins are allowed to access their resources. CORS headers enable secure communication between different domains while protecting against potential cross-site scripting (XSS) and cross-site request forgery (CSRF) attacks.

For example:

Access-Control-Allow-Origin: <https://www.example.com>

Access-Control-Allow-Methods: GET, POST, OPTIONS

Access-Control-Allow-Headers: Content-Type, Authorization

Access-Control-Allow-Credentials: **true**

Access-Control-Expose-Headers: Content-Length, X-Custom-Header

• Access-Control-Allow-Origin: Specifies the allowed origins for cross-origin requests.
• Access-Control-Allow-Methods: Indicates the allowed HTTP methods for cross-origin requests.
• Access-Control-Allow-Headers: Lists the allowed headers for cross-origin requests.
• Access-Control-Allow-Credentials: This header is used in CORS (Cross-Origin Resource Sharing) to indicate whether the server allows credentials (such as cookies, HTTP authentication, or client-side SSL certificates) to be included in cross-origin requests

When a cross-origin request is made, the browser includes an "Origin" header in the request, indicating the domain from which the request originated. The server, in turn, responds with CORS headers to indicate whether the requested resource is allowed to be accessed by the requesting domain.

If the server does not include the necessary CORS headers or includes headers that do not allow the requested domain, the browser blocks the request, and you may encounter a CORS header error. The specific error message will depend on the browser and the circumstances, but some common CORS errors include:

1. "Access-Control-Allow-Origin" header missing: This error occurs when the server does not include the "Access-Control-Allow-Origin" header in the response. This header specifies the allowed origins that can access the resource. If the header is missing or does not include the requesting domain, the browser blocks the request.
2. "Access-Control-Allow-Origin" header has an incorrect value: This error occurs when the server includes the "Access-Control-Allow-Origin" header, but the value does not match the requesting domain. The header should either be "*" (allowing all origins) or the specific origin that is making the request.
3. "Access-Control-Allow-Methods" or "Access-Control-Allow-Headers" headers missing: These errors occur when the server does not include the necessary headers to indicate the allowed HTTP methods (e.g., GET, POST, etc.) or headers that can be included in the request.

You can read detailed articles on debugging and troubleshooting CORS errors by Requestly.

5. Proxy Headers

These headers are used when requests or responses pass through intermediary proxies. These headers allow the client, proxy, or server to exchange information about the origin of the request or to convey information for debugging or logging purposes. These headers provide information or instructions related to proxy behaviour, routing, or client-server interaction.

For example:

Forwarded: for=192.0.2.1; proto=https

X-Forwarded-For: 192.0.2.1, 198.51.100.5

X-Forwarded-Proto: https

• Forwarded: Contains information about the client's IP address and protocol when requests are forwarded through proxies.
• X-Forwarded-For: Indicates the original client IP address when requests are forwarded through proxies.
• X-Forwarded-Proto: Specifies the original protocol (HTTP or HTTPS) when requests are forwarded through proxies.

6. Debugging Headers

These headers are used for debugging and diagnostic purposes. These headers provide additional information about the request or response that can aid in troubleshooting, performance analysis, or monitoring of web applications. They are primarily intended for developers and administrators during the development or debugging phase.

These headers allow developers to associate a token or identifier with a particular debugging session or request. The token can be used to retrieve additional information, such as logs, traces, or detailed debugging data, from a dedicated debugging endpoint or tool.

For example:

X-Debug-Token: abcd1234

X-Debug-Token-Link: /debug/abcd1234

• X-Debug-Token: Contains a token or identifier for retrieving detailed debugging information.
• X-Debug-Token-Link: Provides a link to access the detailed debugging information.

7. Content-Related Headers:

These headers in HTTP provide information about the message body or content of an HTTP request or response. These headers describe various aspects of the content, such as its type, length, encoding, language, and disposition. These content-related headers play a crucial role in how the content is interpreted, processed, and displayed by the client or server. They provide important metadata about the content, allowing recipients to understand its type, length, language, encoding, and how to handle it appropriately.

For example:

Content-Length: 1024

Content-Language: en-US

Transfer-Encoding: chunked

• Content-Type: Specifies the media type or MIME type of the content in the message body.
• Content-Length: Indicates the length of the message body in bytes.
• Content-Encoding: Specifies the encoding applied to the message body, such as gzip or deflate.
• Content-Language: Specifies the language of the content in the message body.
• Content-Disposition: Provides instructions on how the content should be handled or displayed by the recipient.
• Transfer-Encoding: Specifies the encoding applied to the message body during transfer.

Manipulating HTTP Headers

Manipulating HTTP headers refers to the process of modifying or controlling the headers that are sent or received in an HTTP request or response. HTTP headers are key-value pairs that provide additional information about the request or response, such as the content type, caching directives, authentication credentials, and more.

Manipulating HTTP headers can be done for various purposes, such as:

1. Customizing behavior requests: You can modify headers to provide specific information or instructions to the server. For example, you may set custom headers to control caching behavior, specify the preferred language, or provide authentication credentials.
2. Security and privacy: By manipulating headers, you can enhance security and privacy measures. For instance, you can enforce HTTPS connections by adding the "Strict-Transport-Security" header or prevent cross-site scripting attacks by setting the "Content-Security-Policy" header.
3. Content manipulation: Headers can be used to modify the behavior or content of the response. For instance, you can specify compression methods using the "Accept-Encoding" header or request a specific content type using the "Accept" header.

To manipulate HTTP headers, you typically have two approaches:

1. Client-side manipulation: If you are the client making an HTTP request, you can modify the headers before sending the request. This can be done through programming languages like JavaScript and Python or using browser extensions or developer tools.
2. Server-side manipulation: If you are the server handling the request, you can manipulate the headers before sending the response. This can be done in the server-side code, such as PHP, Java, and Node.js, or by using web server configuration files like .htaccess (Apache) or web.config (IIS).

To know how you can modify HTTP headers using Requestly, have a look at this Modify HTTP headers article by Requestly.

‍