How to read response from http.client.HTTPResponse in response hook for URLLib instrumentation #3469

nabheet · 2025-05-05T22:00:53Z

nabheet
May 5, 2025

I am trying to use this hook to log the response for certain status codes. I cannot figure out how to correctly read the response in the response hook for URLLib instrumentation as defined here. If I consume the stream in this response hook, it returns an empty string where this response is actually used. Wondering if anyone has been able to get this work correctly?

Answered by emdneto

May 15, 2025

In that case, you should buffer your response.read() and later restore it. This will happen even if not using the URLLibInstrumentor.

something like: response.read = lambda amt=None: io.BytesIO(body).read(amt)

View full answer

emdneto · 2025-05-07T15:38:54Z

emdneto
May 7, 2025
Collaborator

I tested it and the response_hook is working fine. Create an issue with the repro if you think it's a bug. Thanks

from http.client import HTTPResponse
from urllib.request import Request
from opentelemetry.instrumentation.urllib import URLLibInstrumentor
from opentelemetry.trace import Span
import urllib.request

def response_hook(span: Span, request: Request, response: HTTPResponse):
    print("Response status code:", response.read())


URLLibInstrumentor().instrument(
    response_hook=response_hook
)


url = "http://example.com"
request = Request(url)
urllib.request.urlopen(request)

4 replies

nabheet May 7, 2025
Author

Yes it works. But once you do response.read() it consumes the stream. So your urllib.request.urlopen(request) does not return any data in our code.

Basically, you need another print after your last line of code:

url = "http://example.com"
request = Request(url)
response = urllib.request.urlopen(request)
print("Response status code:", response.read())

This will not print the response content.

This is what I get:

Response status code: b'<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset="utf-8" />\n    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />\n    <meta name="viewport" content="width=device-width, initial-scale=1" />\n    <style type="text/css">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href="https://www.iana.org/domains/example">More information...</a></p>\n</div>\n</body>\n</html>\n'
Response status code: b''

the second print does not have the content from the response as the response is empty.

emdneto May 15, 2025
Collaborator

In that case, you should buffer your response.read() and later restore it. This will happen even if not using the URLLibInstrumentor.

something like: response.read = lambda amt=None: io.BytesIO(body).read(amt)

Answer selected by emdneto

nabheet May 15, 2025
Author

Can you please provide a potential/possible example?

emdneto May 15, 2025
Collaborator

Updated my comment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to read response from http.client.HTTPResponse in response hook for URLLib instrumentation #3469

{{title}}

Replies: 1 comment 4 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

How to read response from http.client.HTTPResponse in response hook for URLLib instrumentation #3469

nabheet May 5, 2025

Replies: 1 comment · 4 replies

emdneto May 7, 2025 Collaborator

nabheet May 7, 2025 Author

emdneto May 15, 2025 Collaborator

nabheet May 15, 2025 Author

emdneto May 15, 2025 Collaborator

nabheet
May 5, 2025

Replies: 1 comment 4 replies

emdneto
May 7, 2025
Collaborator

nabheet May 7, 2025
Author

emdneto May 15, 2025
Collaborator

nabheet May 15, 2025
Author

emdneto May 15, 2025
Collaborator