;jQ d dl Z d dlZd dlmc mZ d dlZd dlZd dlm Z d dl
m Z m Z m
Z
d dlmZ d dlmZ d dlmZmZ G d d Z G d d
e Z G d d e Z G d
de Zd Z G d d ZdS ) N)
ExpatError)TupleOptionalDict)Version)FormattedException) run_commandExternalProgramFailedc e Zd ZdZdZdZdS )GovernorStatusenablederrordisabledN)__name__
__module__ __qualname__ENABLEDERRORDISABLED /builddir/build/BUILDROOT/alt-python27-cllib-3.4.44-1.el8.cloudlinux.x86_64/opt/cloudlinux/venv/lib/python3.11/site-packages/clcommon/lib/mysql_governor_lib.pyr r s G EHHHr r c e Zd ZdS )MySQLGovExceptionN)r r r r r r r r s Dr r c " e Zd ZdZ fdZ xZS )MySQLGovernorDisabledz<
Exception raised when dbgovernor daemon is offline
c V t ddddd d S )NzR%(util)s is disabled in the system. Please, run "%(command)s" to start the servicezMySQL governorzservice db_governor restart)utilcommandmessagecontextsuper__init__self __class__s r r% z MySQLGovernorDisabled.__init__& sK
C )8
r r r r __doc__r%
__classcell__r( s @r r r " sB r r c " e Zd ZdZ fdZ xZS )MySQLGovernorAbsentz:
Exception raised when dbgovernor isn't installed
c T t dddid d S )Nz %(util)s not present in systemr Governorr r# r& s r r% z MySQLGovernorAbsent.__init__6 s>
7
+
r r) r, s @r r. r. 2 sB r r. c dd| idS )NzInvalid %(param)s parameterparamr r )r2 s r _get_exc_messager3 = s 0U# r c
z e Zd ZdZdZdZdZdZdZdZ dZ
d Z d
Z e
j d Zd Zd
Zd Zd>dZd Zd Zd Zed Zd Zd Zd Zed Z ed Z d Z d Z d>d e d e!e"e"f fd Z#d Z$d Z%d! Z&d" Z'd?d$Z(d%e)e* d e+fd&Z,d'e-d(e-d)e-d e!e"e"f fd*Z.d@d%e)e* d e)e/ fd+Z0d,e*d-e-fd.Z1d%e*d/e)e" d0e)e" fd1Z2d%e*d/e)e" d0e)e" fd2Z3 d?d%e*d/e)e" d0e)e" fd3Z4d4 Z5d5 Z6d6 Z7ed7e*d8e*fd9 Z8d>d:Z9d;e d s1vv@ . .
&,,s- 8
B
AB
B")B BBc | } t |dd 5 }t j | cd d d S # 1 swxY w Y d S # t
t f$ r/}d| _ t d|t | dd |d }~wt $ r/}d| _ t d|t | dd |d }~ww xY w) Nrzutf-8)encodingTzNAn error occured while loading governor config from %(path)s. Error: %(error)s)r= r r zNAn error occured while parsing governor config from %(path)s. Error: %(error)s) _get_config_pathopenxml parseStringreadIOErrorOSErrorr; r rb r )r' config_pathfrc s r rL z MySQLGovernor._get_xml_config sc ++-- k3999
1Qqvvxx00
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1! &*D ## E%03q66 B BD D
&*D ## E%03q66 B BD D
sF A( &A A( A A( "A #A( (C 9*B##
C 0*CC c v g } | d d }n'# t $ r}t dddid |d}~ww xY w| d }|D ]Z}| d }|d k r=| d
r(| | d
[|S ) z
Loads information about igrored users
:type governor_cfg: xml.Document
:return: list of ignore users
governorr zFMalformed mysql-governor config. Unable to find element '%(element)s'.elementr Nusermodeignorename)rM rO r rN append)rP ignore_users_listgov_datarc users_data_list user_data user_modes r _load_ignore_users_from_xmlz)MySQLGovernor._load_ignore_users_from_xml s #88DDQGHH # E %z2% %
#77??( M MI!..v66IH$$ ))&11 M%,,Y-C-CF-K-KLLL s
A?Ac | d r|dz }d}n |}|dk rd}d}|| t |dz | j z }|| t |dz | j z }||fS )z
Calculate db R/W IO limits based on governor version
:param io: requested limits in MB/s
:return: string values suitable to pass to "dbctl set ..."
z1.2-17i z%sb,%sb,%sb,%sb z %s,%s,%s,%s)_is_governor_newer_then _percentageint IO_PERCENTS)r' io io_limits limits_tmplrk writes r _calc_rw_io_limitsz MySQLGovernor._calc_rw_io_limits s ' ' 1 1 (W I+KKIA~~ 'KT - -c)q..A.A.2.> @ @ @d . .s9>/B/B/3/? A A AU{r c | sd S t j | j dgd }| d d }|S # t j t f$ r Y d S w xY w)Nz --versionT)text r )r`
subprocess check_output_GOVERNOR_BINARY_PATHstripsplitCalledProcessErrorrm )r' resversions r rC z"MySQLGovernor.get_governor_version s '')) 4 )4+E{*SZ^___C iikk'',,Q/GN-w7 44 s A A$ $A>=A>c X | j }|sdS t | t | k S r8 )rD r )r' r currents r r z%MySQLGovernor._is_governor_newer_then s2 ( 5w ''"2"222r c > d | D S )a
Convert data line from dbctl to list
:param line:
Data line could be like:
"default 400/380/350/300 953/791/724/562 953/791/724/562"
or:
"default 400/380/350/300 1000/830/760/590 1000/830/760/590"
depending on --kb/mb/bb option passed to dbctl
:return: list: ['default', '400/380/350/300', '953/791/724/562', '953/791/724/562']
c g | ]}||S r r ).0parts r
z-MySQLGovernor._parse_line.. s 6666666r r )lines r _parse_linezMySQLGovernor._parse_line s 76 6666r c z | g}|D ]'}| t | |z (t | S )zX
Calculate full list of governor limits by one value and percents koeff
)rw r tuple)valuepercentsr ks r r zMySQLGovernor._percentage
sE
g % %A JJs57|| $ $ $ $Szzr c | j S )z=
Get governor presence flag
:return:
)r@ rE s r r` z!MySQLGovernor.is_governor_present s
((r c D | || j v rdS dS )z
Get MySQL governor status for supplied user
:param username: Username for get status
:return: Governor status: "watched"/"ignored" or None if error
ignoredwatched)ra r9 r' usernames r get_governor_status_by_usernamez-MySQLGovernor.get_governor_status_by_username s,
t3 3 3 9yr with_package_markreturnc | || j v r| j | \ }}}}n| j d \ }}}}|r||f}n||f}|S )a
Get MySQL governor limits for supplied user
:param username: Username for read limits
:param with_package_mark: False - without package limits difference mark (for compatibility with non-package
governor, used in cloudlinux-top/cloudlinux-statistics),
True - with package limits difference mark
:return: Tuple (CPU limit, IO limit).
Examples:
(150, 3072) - with_package_mark == False
('*150', '3072') - with_package_mark == True
('*150', '*4096') - with_package_mark == True
* - user has individual limits, differ from package limit
MySQLGovException will be thrown if governor not present or error
default)ra r: )r' r r user_cpu_limituser_io_limit_kbuser_cpu_limit_markeduser_io_limit_markedlimits_for_returns r get_limits_by_userz MySQLGovernor.get_limits_by_user' s
t, , ,%h/
ZN ,.CEYEY %i0
ZN ,.CEY C!68L M !/1A B r c | |rdnd}| ||g \ }}}|s|rd||p|dd}t | dS )z
Set MySQLGovernor status for single user
:param: `str` username: Username for set status
:param: `bool` status: True for "monitor", False for "ignore"
:return: `bool`: operation status result
monitorru z.Set governor status error(%(ret)s): %(output)sretrT r r ra r[ r r' r status
status_cmdr rY rZ exc_messages r set_governor_status_for_userz*MySQLGovernor.set_governor_status_for_userD s
"(6YYh
$ 9 9:x:P Q Q Wg 1c 1&V.1W=O&P&PR RK#K00 0qr c | |rdnd}|dv rd}| ||g \ }}}|s|rd||p|dd}t | dS ) z
Set user restricted with dbctl utility
:param: `str` username: Username for set restricted status
:param: `bool` status: True for "restricted", False for "unrestricted"
:return: `bool`: operation status result
restrict
unrestrictrootadminr z,Set user restrict error(%(ret)s): %(output)sr r r r r s r set_restricted_status_for_userz,MySQLGovernor.set_restricted_status_for_userT s
#);ZZ|
( ( ( H $ 9 9:x:P Q Q Wg 1c 1&T.1W=O&P&PR RK#K00 0qr c | | dg \ }}}|s|rd||p|dd}t | dS )zi
Set user restricted with dbctl utility
:return: `bool`: operation status result
zunrestrict-all:Set all users unrestrict status error(%(ret)s): %(output)sr r r r )r' r rY rZ r s r %set_unrestricted_status_for_all_usersz3MySQLGovernor.set_unrestricted_status_for_all_usersf su
$ 9 9;K:L M M Wg 1c 1&b.1W=O&P&PR RK#K00 0qr c P | |dv rd}|| j v rdS dS )z
Get MySQL governor status for supplied user
:param username: Username for get status
:return: Governor restricted status: "restricted"/"unrestricted"
r r
restricted unrestricted)ra _governor_restricted_usersr s r get_restrict_status_by_usernamez-MySQLGovernor.get_restrict_status_by_usernames s=
( ( ( H t6 6 6 < ~r Nc 0 ||dS | d|g}|t |t rBd t t
| || j }n t t d |
d| |zt |t r| | \ }}n t t d |
d| |
d | | | \ }}} n/# t $ r"}
t t |
|
d}
~
ww xY w| s|rd
|| p|d d } t | d| _ dS )
a
Set MySQLGovernor limits for user
:param: username `str`: username for set limits
:param: `int`|`list` cpu: governor cpu limit. when it param int -
calculate by percentage other params
:param: `int`|`list` io: io value means that read and write limits similar
:param: `int`|`list` read: read limit
:param: `int`|`list` write: write limit
:return: 0
Nr set,cpu--cpu=r --read=--write=r r r )ra
isinstancer joinmaprb r CPU_PERCENTSr r3 rw r r[ r
r: ) r' r r r cmdrk r r rY rZ rc r s r set_limits_for_userz!MySQLGovernor.set_limits_for_user s ;2:1 h ?#s##
Ahhs3(8(8d>O(P(P Q QRR'(8(?(?@@@
JJ~~~ & & &
> "c""
@ " 5 5b 9 9 ee'(8(>(>???
JJ''' ( ( ( JJ)%)) * * * 3$($=$=c$B$B !C''$ 3 3 3#CFF++ 2 3 1c 1&b.1W=O&P&PR RK#K00 0 $ qs )E
E/
E**E/ package_namec |r4| j dd| d dg}n | j dddg}t |d \ }}}|dk s|r(t d d
| |d d t
j | }n@# t j f$ r-}t d
| j t | dd |d}~ww xY w|S )
Retrieve MySQL Governor package limits
:param package_name: Package name. If None, get all packages name
:return: Dict with limits. Example:
{'pack1': {'cpu': 100, 'io': 900}, 'pack2': {'cpu': 100, 'io': 900}}
getz
--package=zunicode-escapez --format=kbz--allTrR r z''%(cmd)s' failed, stderr is: %(stderr)sr r stderrr z,%(util)s output invalid, error is: %(error)s)r r N)
_PACKAGE_UTILITY_PATHencodedecoder r r jsonloadsJSONDecodeErrorrb )r' r cmd_listrX rY rZ package_data_from_utilrc s r $_get_package_raw_limits_from_utilityz2MySQLGovernor._get_package_raw_limits_from_utility s3 S 2EU\%8%8%:%:%A%ABR%S%SUU%'HH 2E7MRH%0d%S%S%S"'7 q==G=#D#&88H#5#5II% %
%)Z%8%8 " "$' #I$($>QPP% %
&%s B C-(CCcpu_limits_listread_limits_listwrite_limits_listc t |d | t |d t |d fS )a
Calculate package limits from raw governor limits
:param cpu_limits_list: CPU limits list
:param read_limits_list: Read limits list
:param write_limits_list: Write limits list
:return: Tuple: (cpu_limit, io_limit)
r )r _get_user_io_limitrb )r' r r r s r _calc_package_limits_from_rawz+MySQLGovernor._calc_package_limits_from_raw sK ?1%&&(?(?DTUVDW@X@XZ]^opq^rZsZs(t(tttr c | | }n# t $ r w xY wi }| D ]9\ }}| |d |d |d \ }}||d||< :|S )r r rk r )r r )r r itemsr )r' r r
packages_data pack_name pack_limits cpu_limitio_limits r get_package_limitsz MySQLGovernor.get_package_limits s %)%N%N|%\%\ " "
&<&B&B&D&D J J "I{"&"D"D[QVEWYdekYlEPQXEY#[ #[ Ix/8'I'IM) $ $ s $r limit_namesc d | }| j dd| d| g}t |d \ }}}|dk s|r(t dd | |d
d d S )
z
Reset users limits to default
:param username: User name ro reset limits
:param limit_names: Limit names list to reset
r reset_individualz--user=z --limits=TrR r z*'%(cmd)s' is failed, stderr is: %(stderr)sr r r N)r r r r )r' r r
limits_stringr rX _rZ s r reset_user_limits_to_defaultsz+MySQLGovernor.reset_user_limits_to_defaults s --
.0BDXhDXDX/
//1 *8 M M M !W q==G=#G#&88H#5#5II% %
=r r r c ddl m}m} | j | sdS t
j | j | }t
j t
j
| | j k rdS || }n# t t f$ r Y dS w xY wt | dk rdS g }d} |D ]}
|
d r||
d } t | dk rU|dk r| j } nt% | } | d }
|
| k r | d | d
d } | |
d
|
d r||dk r| j }nt% | }|
d } t | dk r7| d }||k r | d
| d
d } t| |
d
| |
| r |||d dS dS )z
Set MySQL Governor to cPanel package file. If limits not changed, package file will not be written
:param package_name: Package name
:param cpu_limit: MySQL CPU limit to set
:param io_limit: MySQL IO limit to set
r )get_file_lineswrite_file_linesNF
lve_mysql_cpu=r r zlve_mysql_cpu=
T lve_mysql_ioz
lve_mysql_io=w)clcommon.utilsr r _PACKAGE_NAME_REmatchr< r= r _CPANEL_PACKAGES_DIRdirnamerealpathrm rl len
startswithr r
_S_DEFAULTrb rw )r' r r r r r package_pathcpanel_package_lineslines_to_writeis_change_mader parts s_cpu_limits_old_cpu_limit
s_io_limitr s r &_set_governor_limits_to_cpanel_packagez4MySQLGovernor._set_governor_limits_to_cpanel_package s DCCCCCCC $**<88 Fw||D$=|LL
7??27++L99 : :d>W W W F #1>,#?#? # FF # $ $ ) ) F ) , ,D//
,I4I
**3//u::?? >>"&/KK"%i..K"'(.."2"2"k11"))*J;*J*J*JKKK%)NN #))T+++6666 00
,X5Iq==!% JJ!$X J
**3//u::?? a((J&&"))*H**H*H*HIII%)NN #))T+++6666 %%d++++ @ \>3 ? ? ? ? ? @ @s B B32B3c ` ddl m} | dk rdS | ||| dS )a&
Apply all MySQL Governor packages limits to cpanel package file. In not cPanel, do nothing
:param package_name: Package name to update
:param cpu_limit: MySQL CPU limit to set, None - not change
:param io_limit: MySQL IO limit to set, None - not change
r ) getCPNamecPanelN)clcommon.cpapir r )r' r r r r s r _apply_package_limits_to_cpanelz-MySQLGovernor._apply_package_limits_to_cpanelV sJ -,,,,, 9;;( " " F 33L)XVVVVVr c & |||t d| d| d| d | j dd|g}|t |t rBd t t | || j }n t t d |
d
| |zt |t r| | \ }}n t t d |
d | |
d
| t |d \ }} }
|dk s|
r)t dd | | |
dd |
||| dS )z
Set limits for Governor package
:param package_name: Package name for set
:param cpu_limit: MySQL CPU limit to set
:param io_limit: MySQL CPU limit to set
NzMySQLGovernor.set_package_limits arguments error: Package name and at least one limit should be provided. Current arguments: package name: z; cpu limit is z; IO limit is ;r z --packager r r r r r TrR r zI'%(command)s' is failed, stdout is: '%(stdout)s', stderr is: '%(stderr)s'r )r stdoutr r )r r r r r r rb r r r3 rw r r r ) r' r r r r r rk r rX rY rZ s r set_package_limitsz MySQLGovernor.set_package_limitsb s 9#49I# %Y\h%Y %Y 5>%Y %Y NV%Y %Y %Y Z Z
Z
.{LQ )S))
Ghhs3(8(8DDU(V(V W WXX'(8(E(EFFF OONSNN + + + (C((
F " 5 5h ? ? ee'(8(D(DEEE OO ,d , , - - - OO .u . . / / /%0d%S%S%S"'7 q==G=#f'*xx'9'9WX_``% %
,,\9hOOOOOr c f t j | j r| j S | j S )zN
Get config path for mysql-governor;
:rtype: str|None
)r< r= r> _CONTAINER_PATH_V2_CONTAINER_PATHrE s r rg z MySQLGovernor._get_config_path s/
7>>$1 2 2 +* *##r c | }| | | _ dS # t $ r d| _ d| _ w xY w)z*Load ignore users list from container fileN)rL r} r9 r r: )r' governor_xmls r _read_ignore_usersz MySQLGovernor._read_ignore_users se //11L00 >>
( ( ( $(D !+/D ( s .2 A c \ | j | j dS | j sd| _ t dd| j id}| | \ }}| dgd \ }}}d |
d d
d D | _ |
d }| ||| | _ d | j vr4d| _ d| _ d| _ d d
| j ddd}t | dS )zL
Loads users info from MySQL governor
:return: None
NTz %(utility)s output is invalidutilityr zlist-restrictedrW c B g | ] }| d S )r r )r r s r r z,MySQLGovernor._load_info.. s1 +
+
+
$DJJLLO+
+
+
r r r r z'There is no %(what)s found in %(where)szdefault settingsz output)whatwhere)r9 r: r@ r; r. r? r& _run_dbctl_listr[ r r r _parse_dbctl_data_linesr )r' utility_exc_messageis_kb_limits_ok gov_data_strr gov_restricted_strgov_data_linesr s r ra zMySQLGovernor._load_info s ' 38M8Y F ( (&*D #%'' '*I+4d6H*I K K
!!! )-(<(<(>(>% % %'8&94 % P P ! q+
+
(:(@(@(B(B(H(H(N(Nqrr(R+
+
+
' &++D11 $ < <^_^q r r D 1 1 1&*D #$(D !+/D (D$6DDVA_A_A_`` K $K00 0 2 1r
read_limit write_limitc t | t | z }nH# t $ r; | |cxk rdk rn nd}n%|dk rt | }nt | }Y nw xY w|S )z
Calculates the io limit.
Handles the situation when user's write or read io limit is less than 1mb/s (PTCLLIB-85).
:type write_limit: str
:type read_limit: str
:rtype: int
z<1r )r
ValueError)r4 r5
user_io_limits r r z MySQLGovernor._get_user_io_limit s 1
OOc+.>.> >MM 1 1 1[0000D00000 !
$$ #J
#K 0 0
1 s " AA'&A'c | ddgd \ }}}d}|sd|v r | dgd \ }}}d}||fS )aW
Executes dbctl list-marked --kb or dbctl list-marked
:param _is_incorrect_syntax: True is emulate dbctl error. Only for testing!
:return: Cortege (is_kb_limits_ok, stdout_str), where
is_kb_limits_ok == True, if dbctl returned limits in KB, else - False
stdout_str - dbctl stdout string
z list-markedz--kbTr) zIncorrect syntaxrU F)r[ )r' _is_incorrect_syntaxrX r1 r r0 s r r- z MySQLGovernor._run_dbctl_list s %)$=$=
F #T %> %; %;!, $#5 #E#E!%!:!:$ "; "0 "0 A|Q #O ,,r r0 r/ c i }|D ]}| }|rd|v r t | }t | dk st |d dk r$d| _ d| _ d| _ t | |d d }|d d }|d d } t | dk s&t | dk st | dk r$d| _ d| _ d| _ t | |d
}
t |d
} |
|d
| d
} |r| n| d z }
|d }|d
d k r d
|d
}n|d
}|d d k rd
|
}nt |
}| |
||f||
< |S )a
Converts data lines from dbctl stdout to dictionary
:param data_lines_list: List of lines from dbctl stdout
:param is_kb_limits_ok: Is limits already in KB/s
:param utility_exc_message: Message dict for exception
:return: Tuple(dict, dict) dbctl data dictionary. Example:
{'default': (400, 1953124, '400', '1953124'),
'cltest1': (350, 2025138, '*350', '2025138')
}
zcpu(%) r TNr / r i +*) r r5 r r r; r: r9 r r r r rb )r' data_lines_listr0 r/ governor_limitsr user_limits_listr r r user_namer r8 r marksr r s r r. z%MySQLGovernor._parse_dbctl_data_lines s( # , y , yD::< > #$$))S1A!1D-E-E-J-J*.'(,%/3,'(;<<< .q 1 7 7 < > ?##q((C0@,A,AQ,F,F#N_J`J`deJeJe*.'(,%/3,'(;<<<(+I !3 4 4N 3 34DQ4GIZ[\I] ^ ^M0? W}}]SWEW %Q'EQx3(@OA,>(@(@%%(7(:%Qx3'=+;'='=$$'*+;'<'<$*8:JLacw)xOI & & r )F)NNrG );r r r r* r r r? r r r# r" r r recompiler r% rH rA r[ r^ r] rL staticmethodr} r rC r r r r` r boolr r r r r r r r r rb dictr rU r r r r r r r rg r& ra r r- r. r r r r5 r5 D s %K%L%M U 39O=J1!rz"677= = =$# # # * * * *,. . .
-
-
- * ! ! \!2 2
3 3 3 7 7 \ 7 \) ) )
! !d !uUXZ]U] ! ! ! !: $ B B B BH&# &SW & & & &8 uT uUY u9= uBGS/ u u u u x} QU &c &B@B@ C=B@ 3- B@ B@ B@ B@H
WC
WHUXM
Wemnqer
W
W
W
W PT59(P (Ps (Px} (P%-c](P (P (P (PT$ $ $ '1 '1 '1R s \ &- - - -&9 9 cg 9 lp 9 9 9 9 9 9 r r5 )r< rG xml.dom.minidomdomminidomri r r xml.parsers.expatr typingr r r packaging.versionr clcommon.clexceptionr clcommon.utils_cmdr r
r r r r. r3 r5 r r r rT s
( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( % % % % % % 3 3 3 3 3 3 A A A A A A A A *
-
+ e e e e e e e e e e r