3
nNjGC @ s d dl mZ d dlZd dlZd dlZddlmZmZ ddlm Z m
Z
ddlm Z ddl m
Z
mZmZ ddlmZ dd lmZ dd
lmZ dd lmZ dd lmZ d
ddgZeje Z d6Z d7Z ej d/e Z!d0d1 Z"ej#e"e!ej#e"e!d2Z$e e
d2Z%G d3d
d
eZ&G d4d de&Z'd5d Z(dS )8 )absolute_importN )HTTPHeaderDictRecentlyUsedContainer)HTTPConnectionPoolHTTPSConnectionPool)port_by_scheme)LocationValueError
MaxRetryErrorProxySchemeUnknown)six)urljoin)RequestMethods) parse_url)Retry PoolManager ProxyManagerproxy_from_urlkey_file cert_file cert_reqsca_certs ssl_version ca_cert_dir ssl_context
key_schemekey_hostkey_port key_timeout key_retries
key_strict key_blockkey_source_address key_key_file
key_cert_file
key_cert_reqs key_ca_certskey_ssl_versionkey_ca_cert_dirkey_ssl_context key_maxsize key_headers
key__proxykey__proxy_headerskey_socket_optionskey__socks_optionskey_assert_hostnamekey_assert_fingerprintkey_server_hostnamePoolKeyc C s |j }|d j |d< |d j |d< x4d D ],}||kr.|| dk r.t|| j ||< q.W |jd}|dk r|t||d< x&t|j D ]}|j||d| < qW x | j D ]}||krd||< qW | f |S )
a
Create a pool key out of a request context dictionary.
According to RFC 3986, both the scheme and host are case-insensitive.
Therefore, this function normalizes both before constructing the pool
key for an HTTPS request. If you wish to change this behaviour, provide
alternate callables to ``key_fn_by_scheme``.
:param key_class:
The class to use when constructing the key. This should be a namedtuple
with the ``scheme`` and ``host`` keys at a minimum.
:type key_class: namedtuple
:param request_context:
A dictionary-like object that contain the context for a request.
:type request_context: dict
:return: A namedtuple that can be used as a connection pool key.
:rtype: PoolKey
schemehostheaders_proxy_headers_socks_optionsNZsocket_optionsZkey_)r6 r7 r8 )
copylower frozensetitemsgettuplelistkeyspop_fields)Z key_classrequest_contextcontextkeyZ socket_optsZfield rF !/usr/lib/python3.6/poolmanager.py_default_key_normalizer; s
rH )httphttpsc @ sx e Zd ZdZdZd ddZdd Zdd Zd d
d Zd d
Z d ddZ
dd Z d ddZ d ddZ
dd Zd!ddZdS )"r a$
Allows for arbitrary requests while transparently keeping track of
necessary connection pools for you.
:param num_pools:
Number of connection pools to cache before discarding the least
recently used pool.
:param headers:
Headers to include with all requests, unless other headers are given
explicitly.
:param \**connection_pool_kw:
Additional parameters are used to create fresh
:class:`urllib3.connectionpool.ConnectionPool` instances.
Example::
>>> manager = PoolManager(num_pools=2)
>>> r = manager.request('GET', 'http://google.com/')
>>> r = manager.request('GET', 'http://google.com/mail')
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> len(manager.pools)
2
N
c K s8 t j| | || _t|dd d| _t| _tj | _d S )Nc S s | j S )N)close)prF rF rG s z&PoolManager.__init__..)Z dispose_func)r __init__connection_pool_kwr poolspool_classes_by_schemekey_fn_by_schemer9 )self num_poolsr6 rP rF rF rG rO s zPoolManager.__init__c C s | S )NrF )rT rF rF rG __enter__ s zPoolManager.__enter__c C s | j dS )NF)clear)rT exc_typeZexc_valZexc_tbrF rF rG __exit__ s zPoolManager.__exit__c C sf | j | }|dkr | jj }xdD ]}|j|d q"W |dkrXxtD ]}|j|d qDW |||f|S )a
Create a new :class:`ConnectionPool` based on host, port, scheme, and
any additional pool keyword arguments.
If ``request_context`` is provided, it is provided as keyword arguments
to the pool class used. This method is used to actually create the
connection pools handed out by :meth:`connection_from_url` and
companion methods. It is intended to be overridden for customization.
Nr4 r5 portrI )r4 r5 rZ )rR rP r9 rA SSL_KEYWORDS)rT r4 r5 rZ rC Zpool_clsrE kwrF rF rG _new_pool s
zPoolManager._new_poolc C s | j j dS )z
Empty our store of pools and direct them all to close.
This will not affect in-flight connections, but they will not be
re-used after completion.
N)rQ rW )rT rF rF rG rW s zPoolManager.clearrI c C sT |s t d| j|}|p d|d< |s:tj|d j d}||d< ||d< | j|S )a
Get a :class:`ConnectionPool` based on the host, port, and scheme.
If ``port`` isn't given, it will be derived from the ``scheme`` using
``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is
provided, it is merged with the instance's ``connection_pool_kw``
variable and used to create the new connection pool, if one is
needed.
zNo host specified.rI r4 P rZ r5 )r _merge_pool_kwargsr r= r: connection_from_context)rT r5 rZ r4 pool_kwargsrC rF rF rG connection_from_host s
z PoolManager.connection_from_hostc C s, |d j }| j| }||}| j||dS )z
Get a :class:`ConnectionPool` based on the request context.
``request_context`` must at least contain the ``scheme`` key and its
value must be a key in ``key_fn_by_scheme`` instance variable.
r4 )rC )r: rS connection_from_pool_key)rT rC r4 Zpool_key_constructorpool_keyrF rF rG r` s
z#PoolManager.connection_from_contextc
C s` | j jN | j j|}|r |S |d }|d }|d }| j||||d}|| j |< W dQ R X |S )z
Get a :class:`ConnectionPool` based on the provided pool key.
``pool_key`` should be a namedtuple that only contains immutable
objects. At a minimum it must have the ``scheme``, ``host``, and
``port`` fields.
r4 r5 rZ )rC N)rQ lockr= r] )rT rd rC Zpoolr4 r5 rZ rF rF rG rc s
z$PoolManager.connection_from_pool_keyc C s t |}| j|j|j|j|dS )a
Similar to :func:`urllib3.connectionpool.connection_from_url`.
If ``pool_kwargs`` is not provided and a new pool needs to be
constructed, ``self.connection_pool_kw`` is used to initialize
the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs``
is provided, it is used instead. Note that if a new pool does not
need to be created for the request, the provided ``pool_kwargs`` are
not used.
)rZ r4 ra )r rb r5 rZ r4 )rT urlra urF rF rG connection_from_url s z PoolManager.connection_from_urlc C sZ | j j }|rVxF|j D ]:\}}|dkrJy
||= W qR tk
rF Y qRX q|||< qW |S )a
Merge a dictionary of override values for self.connection_pool_kw.
This does not modify self.connection_pool_kw and returns a new dict.
Any keys in the override dictionary with a value of ``None`` are
removed from the merged dictionary.
N)rP r9 r< KeyError)rT overrideZbase_pool_kwargsrE valuerF rF rG r_ s
z PoolManager._merge_pool_kwargsTc K s t |}| j|j|j|jd}d|d< d|d< d|krD| jj |d< | jdk rj|jdkrj|j||f|}n|j||j f|}|o|j
}|s|S t ||}|j dkrd }d|d
< t
|d j |d< |jd } t| tstj| |d } | jo|j| r %s) r rb r5 rZ r4 r6 r9 proxyurlopenZ request_uriZget_redirect_locationr
Zstatusr Z_prepare_for_method_changer=
isinstancer Zfrom_intZremove_headers_on_redirectZ is_same_hostr? r Ziterkeysr: rA Z incrementr
Zraise_on_redirectloginfo) rT methodrf rl r\ rg Zconnrn Zredirect_locationrm r6 headerrF rF rG rp / sH
zPoolManager.urlopen)rK N)N)NrI N)N)N)T)__name__
__module__ __qualname____doc__ro rO rV rY r] rW rb r` rc rh r_ rp rF rF rF rG r { s
c sH e Zd ZdZd fdd Zd fdd Zdd d
Zd fd d
Z ZS )r ax
Behaves just like :class:`PoolManager`, but sends all requests through
the defined proxy, using the CONNECT method for HTTPS URLs.
:param proxy_url:
The URL of the proxy to be used.
:param proxy_headers:
A dictionary containing headers that will be sent to the proxy. In case
of HTTP they are being sent with each request, while in the
HTTPS/CONNECT case they are sent only once. Could be used for proxy
authentication.
Example:
>>> proxy = urllib3.ProxyManager('http://localhost:3128/')
>>> r1 = proxy.request('GET', 'http://google.com/')
>>> r2 = proxy.request('GET', 'http://httpbin.org/')
>>> len(proxy.pools)
1
>>> r3 = proxy.request('GET', 'https://httpbin.org/')
>>> r4 = proxy.request('GET', 'https://twitter.com/')
>>> len(proxy.pools)
3
rK Nc s t |tr d|j|j|jf }t|}|jsFtj|jd}|j|d}|jdkrZt |j|| _
|pfi | _ | j
|d< | j |d< t t
| j||f| d S ) Nz
%s://%s:%ir^ )rZ rI rJ _proxyr7 )rI rJ )rq r r4 r5 rZ r r r= _replacer ro
proxy_headerssuperr rO )rT proxy_urlrU r6 r| rP ro rZ ) __class__rF rG rO s
zProxyManager.__init__rI c sD |dkr t t| j||||dS t t| j| jj| jj| jj|dS )NrJ )ra )r} r rb ro r5 rZ r4 )rT r5 rZ r4 ra )r rF rG rb s
z!ProxyManager.connection_from_hostc C s0 ddi}t |j}|r ||d< |r,|j| |S )z
Sets headers needed by proxies: specifically, the Accept and Host
headers. Only sets headers not provided by the user.
ZAcceptz*/*ZHost)r netlocupdate)rT rf r6 Zheaders_r rF rF rG _set_proxy_headers s
z ProxyManager._set_proxy_headersTc sN t |}|jdkr0|jd| j}| j|||d< tt| j||fd|i|S )z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.rI r6 rl )r r4 r= r6 r r} r rp )rT rt rf rl r\ rg r6 )r rF rG rp s
zProxyManager.urlopen)rK NN)NrI N)N)T) rv rw rx ry rO rb r rp
__classcell__rF rF )r rG r p s
c K s t f d| i|S )Nr~ )r )rf r\ rF rF rG r s )r r r r r r r )r r r r r r r! r" r# r$ r% r& r' r( r) r* r+ r, r- r. r/ r0 r1 r2 ))Z
__future__r collections functoolsZlogging _collectionsr r Zconnectionpoolr r r
exceptionsr r
r Zpackagesr Z packages.six.moves.urllib.parser
Zrequestr Zutil.urlr Z
util.retryr __all__Z getLoggerrv rr r[ Z _key_fields
namedtupler3 rH partialrS rR r r r rF rF rF rG sd
6
vW