python_active_versions.python_active_versions.CachedHTMLSession

Bases: CacheMixin, HTMLSession

Session with features from both CachedSession and HTMLSession.

__init__(cache_name: Path | str = 'http_cache', backend: str | BaseCache | None = None, serializer: str | SerializerPipeline | Stage | None = None, expire_after: None | int | float | str | datetime | timedelta = -1, urls_expire_after: Dict[str | Pattern, None | int | float | str | datetime | timedelta] | None = None, cache_control: bool = False, allowable_codes: Iterable[int] = (200,), allowable_methods: Iterable[str] = ('GET', 'HEAD'), always_revalidate: bool = False, ignored_parameters: Iterable[str] = ('Authorization', 'X-API-KEY', 'access_token', 'api_key'), match_headers: Iterable[str] | bool = False, filter_fn: Callable[[Response], bool] | None = None, key_fn: Callable[[...], str] | None = None, stale_if_error: bool | int = False, **kwargs)

Methods

`__init__`([cache_name, backend, serializer, ...])
`cache_disabled`()	Context manager for temporary disabling the cache
`close`()	Close the session and any open backend connections
`delete`(url, **kwargs)	Sends a DELETE request.
`get`(url[, params])	Sends a GET request.
`get_adapter`(url)	Returns the appropriate connection adapter for the given URL.
`get_redirect_target`(resp)	Receives a Response.
`head`(url, **kwargs)	Sends a HEAD request.
`merge_environment_settings`(url, proxies, ...)	Check the environment and merge it with some settings.
`mount`(prefix, adapter)	Registers a connection adapter to a prefix.
`options`(url, **kwargs)	Sends a OPTIONS request.
`patch`(url[, data])	Sends a PATCH request.
`post`(url[, data])	Sends a POST request.
`prepare_request`(request)	Constructs a `PreparedRequest` for transmission and returns it.
`put`(url[, data])	Sends a PUT request.
`rebuild_auth`(prepared_request, response)	When being redirected we may want to strip authentication from the request to avoid leaking credentials.
`rebuild_method`(prepared_request, response)	When being redirected we may want to change the method of the request based on certain specs or browser behavior.
`rebuild_proxies`(prepared_request, proxies)	This method re-evaluates the proxy configuration by considering the environment variables.
`request`(method, url, *args[, headers, ...])	This method prepares and sends a request while automatically performing any necessary caching operations.
`resolve_redirects`(resp, req[, stream, ...])	Receives a Response.
`response_hook`(response, **kwargs)	Change response enconding and replace it by a HTMLResponse.
`send`(request[, expire_after, ...])	Send a prepared request, with caching.
`should_strip_auth`(old_url, new_url)	Decide whether Authorization header should be removed when redirecting
`wrap`(original_session, **kwargs)	Add caching to an existing `Session` object, while retaining all original session settings.

Attributes

`browser`
`expire_after`
`settings`	Settings that affect cache behavior
`headers`	A case-insensitive dictionary of headers to be sent on each `Request` sent from this `Session`.
`auth`	Default Authentication tuple or object to attach to `Request`.
`proxies`	Dictionary mapping protocol or protocol and host to the URL of the proxy (e.g. {'http': 'foo.bar:3128', 'http://host.name': 'foo.bar:4012'}) to be used on each `Request`.
`hooks`	Event-handling hooks.
`params`	Dictionary of querystring data to attach to each `Request`.
`stream`	Stream response content default.
`verify`	SSL Verification default.
`cert`	SSL client certificate default, if String, path to ssl client cert file (.pem).
`max_redirects`	Maximum number of redirects allowed.
`trust_env`	Trust environment settings for proxy configuration, default authentication and similar.
`cookies`	A CookieJar containing all currently outstanding cookies set on this session.

auth: Default Authentication tuple or object to attach to Request.

cache_disabled()

Context manager for temporary disabling the cache

Warning

This method is not thread-safe.

Example

>>> s = CachedSession()
>>> with s.cache_disabled():
...     s.get('https://httpbin.org/ip')

cert: SSL client certificate default, if String, path to ssl client cert file (.pem). If Tuple, (‘cert’, ‘key’) pair.

close(): Close the session and any open backend connections

cookies: A CookieJar containing all currently outstanding cookies set on this session. By default it is a RequestsCookieJar, but may be any other cookielib.CookieJar compatible object.

delete(url: str, **kwargs) → OriginalResponse | CachedResponse

Sends a DELETE request. Returns Response object.

Parameters:

url – URL for the new Request object.
**kwargs – Optional arguments that request takes.

Return type:

requests.Response

get(url: str, params=None, **kwargs) → OriginalResponse | CachedResponse

Sends a GET request. Returns Response object.

Parameters:

url – URL for the new Request object.
**kwargs – Optional arguments that request takes.

Return type:

requests.Response

get_adapter(url)

Returns the appropriate connection adapter for the given URL.

Return type:: requests.adapters.BaseAdapter

get_redirect_target(resp): Receives a Response. Returns a redirect URI or None

head(url: str, **kwargs) → OriginalResponse | CachedResponse

Sends a HEAD request. Returns Response object.

Parameters:

url – URL for the new Request object.
**kwargs – Optional arguments that request takes.

Return type:

requests.Response

headers: A case-insensitive dictionary of headers to be sent on each Request sent from this Session.

hooks: Event-handling hooks.

max_redirects: Maximum number of redirects allowed. If the request exceeds this limit, a TooManyRedirects exception is raised. This defaults to requests.models.DEFAULT_REDIRECT_LIMIT, which is 30.

merge_environment_settings(url, proxies, stream, verify, cert)

Check the environment and merge it with some settings.

Return type:: dict

mount(prefix, adapter)

Registers a connection adapter to a prefix.

Adapters are sorted in descending order by prefix length.

options(url: str, **kwargs) → OriginalResponse | CachedResponse

Sends a OPTIONS request. Returns Response object.

Parameters:

url – URL for the new Request object.
**kwargs – Optional arguments that request takes.

Return type:

requests.Response

params: Dictionary of querystring data to attach to each Request. The dictionary values may be lists for representing multivalued query parameters.

patch(url: str, data=None, **kwargs) → OriginalResponse | CachedResponse

Sends a PATCH request. Returns Response object.

Parameters:

url – URL for the new Request object.
data – (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the Request.
**kwargs – Optional arguments that request takes.

Return type:

requests.Response

post(url: str, data=None, **kwargs) → OriginalResponse | CachedResponse

Sends a POST request. Returns Response object.

Parameters:

url – URL for the new Request object.
data – (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the Request.
json – (optional) json to send in the body of the Request.
**kwargs – Optional arguments that request takes.

Return type:

requests.Response

prepare_request(request)

Constructs a PreparedRequest for transmission and returns it. The PreparedRequest has settings merged from the Request instance and those of the Session.

Parameters:: request – Request instance to prepare with this session’s settings.
Return type:: requests.PreparedRequest

proxies: Dictionary mapping protocol or protocol and host to the URL of the proxy (e.g. {‘http’: ‘foo.bar:3128’, ‘http://host.name’: ‘foo.bar:4012’}) to be used on each Request.

put(url: str, data=None, **kwargs) → OriginalResponse | CachedResponse

Sends a PUT request. Returns Response object.

Parameters:

url – URL for the new Request object.
data – (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the Request.
**kwargs – Optional arguments that request takes.

Return type:

requests.Response

rebuild_auth(prepared_request, response): When being redirected we may want to strip authentication from the request to avoid leaking credentials. This method intelligently removes and reapplies authentication where possible to avoid credential loss.

rebuild_method(prepared_request, response): When being redirected we may want to change the method of the request based on certain specs or browser behavior.

rebuild_proxies(prepared_request, proxies)

This method re-evaluates the proxy configuration by considering the environment variables. If we are redirected to a URL covered by NO_PROXY, we strip the proxy configuration. Otherwise, we set missing proxy keys for this URL (in case they were stripped by a previous redirect).

This method also replaces the Proxy-Authorization header where necessary.

Return type:: dict

This method prepares and sends a request while automatically performing any necessary caching operations. This will be called by any other method-specific requests functions (get, post, etc.). This is not used by PreparedRequest objects, which are handled by send().

See requests.Session.request() for base parameters. Additional parameters:

Parameters:

expire_after – Expiration time to set only for this request. See expiration for details.
only_if_cached – Only return results from the cache. If not cached, return a 504 response instead of sending a new request.
refresh – Revalidate with the server before using a cached response, and refresh if needed (e.g., a “soft refresh,” like F5 in a browser)
force_refresh – Always make a new request, and overwrite any previously cached response (e.g., a “hard refresh”, like Ctrl-F5 in a browser))

Returns:

Either a new or cached response

resolve_redirects(resp, req, stream=False, timeout=None, verify=True, cert=None, proxies=None, yield_requests=False, **adapter_kwargs): Receives a Response. Returns a generator of Responses or Requests.

response_hook(response, **kwargs) → HTMLResponse: Change response enconding and replace it by a HTMLResponse.

Send a prepared request, with caching. See requests.Session.send() for base parameters, and see request() for extra parameters.

Order of operations: For reference, a request will pass through the following methods:

requests.get(), CachedSession.get(), etc. (optional)
CachedSession.request()
requests.Session.request()
CachedSession.send()
BaseCache.get_response()
requests.Session.send() (if not using a cached response)
BaseCache.save_response() (if not using a cached response)

property settings: CacheSettings: Settings that affect cache behavior

should_strip_auth(old_url, new_url): Decide whether Authorization header should be removed when redirecting

stream: Stream response content default.

trust_env: Trust environment settings for proxy configuration, default authentication and similar.

verify: SSL Verification default. Defaults to True, requiring requests to verify the TLS certificate at the remote end. If verify is set to False, requests will accept any TLS certificate presented by the server, and will ignore hostname mismatches and/or expired certificates, which will make your application vulnerable to man-in-the-middle (MitM) attacks. Only set this to False for testing.

classmethod wrap(original_session: Session, **kwargs) → CacheMixin

Add caching to an existing Session object, while retaining all original session settings.

Parameters:

original_session – Session object to wrap
kwargs – Keyword arguments for CachedSession