ijddlZddlZddlZddlmZddlmZddlmZddl m Z ddl m Z ddl m Z ddl mZdd lmZdd lmZ ed d gGd dZedddej(dfdZedddej(dfdZdZdZedGddeZdZGddeZed d6d Z ed!d7d"Zed#d$Zed%d&Zed'd8d(Z ed)d8d*Z!Gd+d,eZ"ddd-d.d/dd0d1Z#d2Z$d3Z% d9d4Z&d9d5Z'y):N)backend)ops) keras_export) KerasTensor)any_symbolic_tensors)canonicalize_axis)standardize_axis_for_numpy) Operation GPTQConfigzkeras.Quantizerzkeras.quantizers.Quantizerc0eZdZddZdZedZdZy) Quantizerc||_yN output_dtype)selfrs t/var/www/html/emotional.easysim.app/public_html/venv/lib/python3.12/site-packages/keras/src/quantizers/quantizers.py__init__zQuantizer.__init__s (c|S)z0Compute a quantized output from an input tensor.)rxs r__call__zQuantizer.__call__src|di|S)aCreates a quantizer from its config. This method is the reverse of `get_config`, capable of instantiating the same quantizer from the config dictionary. This method is used by Keras `model_to_estimator`, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON. Args: config: A Python dictionary, typically the output of get_config. Returns: A quantizer instance. rr)clsconfigs r from_configzQuantizer.from_configs$}V}rct|d)aReturns the config of the quantizer. A quantizer config is a Python dictionary (serializable) containing all configuration parameters of the quantizer. The same quantizer can be reinstantiated later (without any saved state) from this configuration. This method is optional if you are just training and executing models, exporting to and from SavedModels, or using weight checkpoints. This method is required for Keras `model_to_estimator`, saving and loading models to HDF5 formats, Keras model cloning, some visualization utilities, and exporting models to and from JSON. Returns: Python dictionary. z does not implement get_config())NotImplementedError)rs r get_configzQuantizer.get_config0s$"TF*J"KLLrN)int8)__name__ __module__ __qualname__rr classmethodrr!rrrrrs&)&Mrrz!keras.quantizers.abs_max_quantizeir"Fc 4|rtj|j}tj|}t |}t j|dt jt jt j||d|}t j||}t jt j||d|d}|j|}tj|tj||fStj|}tj|dtjtjtj||d|}tj |tj|j}tj||}tjtj||d|d}tj ||}||fS)a Quantizes the input tensor using the absolute maximum quantization scheme. Args: inputs: Input tensor to quantize. axis: Axis along which to compute the quantization range. value_range: Tuple of the minimum and maximum values of the quantization range. dtype: Data type of the quantized output. epsilon: Small value to avoid division by zero. to_numpy: Whether to perform the quantization in numpy. This performs the computation on the host CPU and can be useful for saving memory on the device. If False, the computation is performed on the device. Returns: A tuple of the quantized tensor and the scale. Taxiskeepdimsrdtype)rstandardize_dtyper/rconvert_to_numpyr npdivideaddmaxabsmultiplycliproundastypeconvert_to_tensorcast) inputsr, value_ranger/epsilonto_numpyoriginal_dtypescaleoutputss rabs_max_quantizerDEs4 226<<@%%f-)$/ N FF266"&&.tdCW M ++fe,''"((7+[^[^L..'$$W-s/D/D 0    " "6 *F JJA dTBGL E HHUG55fllC DEll65)Ghhsyy);q>;q>JGhhw&G E>rz9keras.quantizers.abs_max_quantize_grouped_with_zero_point)icB|rt|||||St|||||S)aQuantizes a 2D tensor using grouped asymmetric quantization with zero point. Groups are formed along axis 0 (the input/contracting dimension). Each group of `block_size` rows gets its own scale factor and zero point per column. This is useful for weight distributions that are not centered around zero. Args: inputs: Input tensor to quantize. Shape: `(input_dim, output_dim)`. block_size: Number of elements per group along axis 0. value_range: Tuple of `(min, max)` quantization range. dtype: Data type of quantized output. epsilon: Small value to avoid division by zero. to_numpy: Whether to perform computation in numpy for memory efficiency. Returns: A tuple `(quantized_tensor, scale, zero_point)` where: - `quantized_tensor`: Same shape as inputs, dtype=`dtype`. - `scale`: Shape `(n_groups, output_dim)` where `n_groups = ceil(input_dim / block_size)`. - `zero_point`: Shape `(n_groups, output_dim)`, dtype=`uint8`. Example: ```python >>> import numpy as np >>> from keras.quantizers import abs_max_quantize_grouped_with_zero_point >>> kernel = np.random.randn(512, 256).astype("float32") >>> quantized, scale, zero_point = abs_max_quantize_grouped_with_zero_point( ... kernel, block_size=128, value_range=(-8, 7) ... ) >>> quantized.shape (512, 256) >>> scale.shape # 512 / 128 = 4 groups (4, 256) >>> zero_point.shape (4, 256) ``` )/_abs_max_quantize_grouped_with_zero_point_numpy0_abs_max_quantize_grouped_with_zero_point_tensor)r= block_sizer>r/r?r@s r(abs_max_quantize_grouped_with_zero_pointrJ{s:d> J UG   < K rctj|j}tj|}|j \}}t j||z }|\} } ||z} | |kDr@tj| |z |f|j} tj|| gd} n|} | j|||}tj|dd}tj|dd}tjtj|||z| | z }tj tj| || z}tj"|| | }tj tj|||z}tj"|| | }|j%|}|j| |d|ddf}tj&|d}tj&|dj%d}tj(|tj(||tj(|fS) zNumPy implementation of grouped asymmetric quantization. Uses NumPy for computation to reduce GPU memory usage during model quantization. r.rr,r*Tr+Nr")rr0r/rr1shapemathceilr2zeros concatenatereshapeminr5r3subtractr9r8r:squeezer;)r=rIr>r/r?rA input_dim output_dimn_groupsqminqmaxpadded_input_dimpadding inputs_paddedinputs_reshapedmin_valmax_valrB zero_pointrCs rrGrGs..v||J g& e>: j) rc ^tj|j}tj|}tj |}|d}|d}|\} } | | z dz} t tj| } t tjt ||z } | |z}tj|}t|| dd|||d\}}}tj|}tj|}|t |z }|dkDr=tj||f|j}tj||gd}n|}tj|| ||f}tj|d}tj|d}tj tj"tj$|||}tj&|| | }tj(||}tj|||f}|d|ddf}|||fS) zATensor backend implementation of grouped asymmetric quantization.rr*FT)bits symmetric per_channel group_size compute_dtyper?signedr.rLN)rr0r/rr;rMintrNlog2rO transposecompute_quantization_parametersrPrQrR expand_dimsr4r9r3r8r<)r=rIr>r/r?rA input_shaperVrWrYrZ num_levelsrcrXr[inputs_tscale_t zero_point_t_rBrapad_sizer\r]r^scale_expandedzero_point_expandedrCs rrHrHs..v||Ghhwd+Ghhw&Gkk'$4j#ABGjyj!m$G E: %%rz keras.quantizers.AbsMaxQuantizercHeZdZddejdfdZddZdZy)AbsMaxQuantizerNr'r"ctj|||$t|tr|f}t ||_nd|_||_||_|dk(r|ddks|ddkDrtd|yy)Nrr"rir*r(zuQuantizer with output_dtype='int8' requires value_range to be within the interval [-128, 127]. Received: value_range=) rr isinstanceritupler,r>r? ValueError)rr,r>r?rs rrzAbsMaxQuantizer.__init__5s 4l;  $$wd DIDI& 6 !1~$ A(< ##.-1)= "rc| |j}|d}t|||j|j|j|\}}||fS)a Quantizes the input tensor. Args: x: Input tensor to quantize. axis: Axis along which to compute the quantization range. If None, uses the axis specified in the constructor. If None and no axis was specified in the constructor, defaults to -1. to_numpy: Whether to perform the quantization in numpy. This performs the computation on the host CPU and can be useful for saving memory on the device. If False, the computation is performed on the device. Returns: A tuple of the quantized tensor and the scale. )r,rDr>rr?)rrr,r@ quantized_xrBs rrzAbsMaxQuantizer.__call__MsZ" <99D <D-        LL    UE!!rc|j|j|jd}|j|j|d<|S)N)r>r?rr,)r>r?rr,)rrs rr!zAbsMaxQuantizer.get_configms@++|| --  99 !YYF6N r)NF)r#r$r%rr?rrr!rrrrxrx3s)! 0"@rrxctj|jd}tj||}tj||}d|zdz }|sdnd}tj ||}tj |||z }tj ||z |} |tj ||z } tj| ||} tj| } tjtj || |} tjtj || |}| ||| fS)z>Adjusts and nudges the quantization range for better accuracy.float32r*r) r result_typer/rr<rTr3r8r9r7) min_range max_rangenum_bits narrow_rangerg quant_max quant_min diff_rangerB inv_scalezero_point_from_minranudged_zero_point nudged_min nudged_maxs radjust_and_nudgerxs'' CMM2IM2Ih!#I%1Ii3J JJz9y#8 9E 9y0*=I$cjjE&BB-y)DJ *-cll96GH%PJcll96GH%PJ z5) 33rc,eZdZdfd ZdZdZxZS)FakeQuantWithMinMaxVarscLt|||_||_||_yr)superrrrr,)rrrr, __class__s rrz FakeQuantWithMinMaxVars.__init__s$   ( rc`t||||j|j|jS)N)rrr,)fake_quant_with_min_max_varsrrr,rr=min_valsmax_valss rcallzFakeQuantWithMinMaxVars.calls/+   ]]**   rcDt|j|jS)Nr.)rrMr/rs rcompute_output_specz+FakeQuantWithMinMaxVars.compute_output_specs6<._fake_quant_with_min_max_vars_per_channel..grad"s!" 5(C0B$S]3A!qDyAADA~~a4Hyy8S9H778$7778,{{8SYYw-?@H((J7Hyy8S9H778$7778,{{8SYYw-?@Hx) )-Bs E E)rr0r/rrfloorr4r7r8r<rT logical_andrr)rr_r`r/rBr quant_zero x_clampedx_clamped_shiftedresultrrrrr,rrs``` @@@r)_fake_quant_with_min_max_vars_per_channelzOfake_quant_with_min_max_vars.._fake_quant_with_min_max_vars_per_channelsC))!''24D Wh 4 0 JyYY GGCLL*i8# > HH HHQ (( ):z  LLJ? IILL %6 BJ     &.   a ,cnnQ .K "& * *<t|r)rr symbolic_callrr;rirndimrrr0r/ quantizationrr<rRswapaxes(fake_quant_with_min_max_vars_per_channelcustom_gradient) r=rrrrr,tfr/rC last_axisrs ``` rrrsJVI&&(66 Hh   " "6 *F$$X.H$$X.H8}H  v{{3L())&,,7 <ooBB+Xr2I>Xr2I>!) CG88G51 1  aI\\&$ :FooNN+9-9-!) OGhhwe4G<<D9 9??B 5VXx PPrz%keras.quantizers.compute_float8_scalec8tj|}tjtj||d|z}tj|dkD||}tjtj|||}tj|S)Nr)r reciprocalr3risfinite)amaxrB dtype_maxmarginsfs rcompute_float8_scalerHsq NN5 !E CJJy$/F ;B 4#:r5 )B 3<<%r5 1B >>" rz,keras.quantizers.compute_float8_amax_historyctjtjtj||j}tj tj |ddggtj|dg}|S)Nr~)shiftrr*)rr<r5r6r/scatter_updaterollrR)r amax_history amax_updatenew_amax_historys rcompute_float8_amax_historyrUsg((3773771:. 0B0BCK)) R(  K!% rz(keras.quantizers.quantize_and_dequantizectjttj|j |}tj |tj||}tj|| |}tj||}tjtj||tj||}|Sr) rr<float ml_dtypesfinfor5r3r8r7)r=rBquantized_dtypergquantized_dtype_maxrs rquantize_and_dequantizer`s(( iooo.223] 6388E=9:A ((*=>A O$A SXXa/%1OPA Hrzkeras.quantizers.pack_int4c V|dvrtd|dtj|j|k7r/t d|dtj|jdt j |}tj|}t|j}|dkr||z }tj||d}|jd}|dzd k(rJd |jd d z}tj|tj||j gd }|d d d}|d d d} tjd| } tj|j|| } tj| j|| } tj | tj"| tjd| } | j|} tj| d|} t j$| }|t'| j|fS)ae Pack an int4 tensor into an int8 tensor with packed nibbles. The input values must already be int8 in the signed range `[-8, 7]` and represent the desired int4 values. Packing is performed along the specified axis (default is 0). For every two consecutive rows, the **low nibble** of the output byte stores the value from the first row, and the **high nibble** stores the value from the second row. Args: arr: An `int8` or `uint8` tensor containing int4 values in the range `[-8, 7]`. axis: The axis along which to pack the tensor. Defaults to 0. dtype: The data type of the input and packed tensor. Can be `"int8"` or `"uint8"`. Defaults to `"int8"`. Returns: tuple: A tuple `(packed, packed_shape, orig_rows)` where `packed` is the packed int8 tensor with int4 values stored in nibbles, `packed_shape` is the shape of the packed tensor, and `orig_rows` is the original (unpacked) row count prior to any padding that may have been inserted when an odd number of rows is supplied. Example: ```python >>> import numpy as np >>> from keras.quantizers import pack_int4, unpack_int4 # Example with axis=0 # Original array has shape (3, 2) >>> original_array = np.array([[-3, 7], [2, -8], [1, 0]], dtype=np.int8) # Pack the array along axis 0. Since the length of axis 0 (3) is # odd, it will be padded to a length of 4. The packed array will # have a shape of (ceil(3/2), 2) = (2, 2). >>> packed, packed_shape, orig_len = pack_int4(original_array, axis=0) >>> print("Packed array: ", packed) Packed array: [[ 45 -121] [ 1 0]] # Now, unpack the array back to its original form >>> unpacked = unpack_int4(packed, orig_len, axis=0) >>> print("Unpacked array: ", unpacked) Unpacked array: [[-3 7] [ 2 -8] [ 1 0]] >>> np.allclose(original_array, unpacked) True # Example with axis=1 # Original array has shape (2, 3) >>> original_array = np.array([[-3, 7, 2], [-8, 1, 0]], dtype=np.int8) # Pack along axis 1. Length of axis 1 (3) is padded to 4. # The new shape is (2, ceil(3/2)) = (2, 2). >>> packed, packed_shape, orig_len = pack_int4(original_array, axis=1) >>> print("Packed array: ", packed) Packed array: [[ 125 2] [ 24 0]] # Unpack the array >>> unpacked = unpack_int4(packed, orig_len, axis=1) >>> print("Unpacked array: ", unpacked) Unpacked array: [[-3 7 2] [-8 1 0]] >>> np.allclose(original_array, unpacked) True ``` r"uint81Expected dtype to be 'int8' or 'uint8', but got ''.z Expected z tensor for packing, got .rrr*)r*Nr.rL)r|rr0r/ TypeErrorrr1r2rrMmoveaxisrQrParray bitwise_andr: bitwise_or left_shiftr;r{)arrr,r/arr_npnp_dtyperankn pad_shapelowhighmasklow_uhigh_u packed_nppackeds r pack_int4rosZ %%?wb I    +u4w7((34A 7   ! !# &FxxH v|| D ax  [[q )F  QA1uz6<<++  RXXiv||< =A  1+C !$Q$>> import numpy as np >>> from keras.quantizers import pack_int4, unpack_int4 # Example with axis=0 # Original array has shape (3, 2) >>> original_array = np.array([[-3, 7], [2, -8], [1, 0]], dtype=np.int8) # Pack the array along axis 0. Since the length of axis 0 (3) is # odd, it will be padded to a length of 4. The packed array will # have a shape of (ceil(3/2), 2) = (2, 2). >>> packed, packed_shape, orig_len = pack_int4(original_array, axis=0) >>> print("Packed array: ", packed) Packed array: [[ 45 -121] [ 1 0]] # Now, unpack the array back to its original form >>> unpacked = unpack_int4(packed, orig_len, axis=0) >>> print("Unpacked array: ", unpacked) Unpacked array: [[-3 7] [ 2 -8] [ 1 0]] >>> np.allclose(original_array, unpacked) True # Example with axis=1 # Original array has shape (2, 3) >>> original_array = np.array([[-3, 7, 2], [-8, 1, 0]], dtype=np.int8) # Pack along axis 1. Length of axis 1 (3) is padded to 4. # The new shape is (2, ceil(3/2)) = (2, 2). >>> packed, packed_shape, orig_len = pack_int4(original_array, axis=1) >>> print("Packed array: ", packed) Packed array: [[ 125 2] [ 24 0]] # Unpack the array >>> unpacked = unpack_int4(packed, orig_len, axis=1) >>> print("Unpacked array: ", unpacked) Unpacked array: [[-3 7 2] [-8 1 0]] >>> np.allclose(original_array, unpacked) True ``` rrrz1Expected int8 or uint8 tensor for unpacking, got ctj|j}tjd|}tj tj |||S)zConverts unpacked nibbles [0, 15] to signed int4 [-8, 7]. Uses a branchless XOR approach: (x ^ 8) - 8 This maps: 0->0, 1->1, ..., 7->7, 8->-8, 9->-7, ..., 15->-1 r)rr0r/rr<rT bitwise_xor)rdtype_xeights r to_signedzunpack_int4..to_signedHsD ++AGG4G$||COOAu5u==rrNrrrr.rr"r*rLr~.)r|rr0r/rgetattrrMrrrr right_shiftr<stackrRr{rindexrk)rorig_lenr,r/rrr low_unpacked high_unpacked low_final high_finalstackedunpackedrperminv_perm transposedrrs r unpack_int4rsZ %%?wb I    .6GG? ~ N  > 6<< . C#fll2CD ax   qyTQYyyV\\2vt4 (BDI F?$\2L%m4MHH\51 XXmU3 ))Y 3!<;;wcii6G6K0L(LM  3'' 6d 91qDyQ9 9D',T{3! 1 3H3vt,J 99T .D //*d +C ??3??:q94 @D n ((3 C 88D% Diid !,G{{7EE#))J2G2K,L$LMH  3'H}}Xx0H O3:3s L)L?LcTeZdZdZedddfdZdZfdZedZ xZ S) GPTQQuantizeraA class that handles the quantization of weights using GPTQ method. This class provides methods to find quantization parameters (scale and zero) for a given tensor and can be used to quantize weights in a GPTQ context. Args: weight_bits: (int) The number of bits to quantize to (e.g., 4). per_channel: (bool) A flag indicating whether quantization is applied per-channel (`True`) or per-tensor (`False`). Defaults to `False`. symmetric: (bool) A flag indicating whether symmetric (`True`) or asymmetric (`False`) quantization is used. Defaults to `False`. group_size: (int) The size of weight groups for quantization. A value of -1 indicates that grouping is not used. Defaults to -1. N) tokenizerdatasetrctj||j|_|j|_|j|_|j |_||_d|_d|_d|_ yr) rr weight_bitsrerdrfrgrBzeromaxq)rrrgs rrzGPTQQuantizer.__init__sg 4 !--!--)) ++*   rct||j|j|j|j|j \|_|_|_|j |j|jfS)zBFinds quantization parameters (scale and zero) for a given tensor.)rcrdrerfrg) rlrrdrerfrgrBrr)r input_tensors r find_paramszGPTQQuantizer.find_paramss`+J !!nn((,, , ( DItyzz499dii//rct|}|j|j|j|j |j d|S)N)rrerdrf)rr!updaterrerdrf)rrrs rr!zGPTQQuantizer.get_configsH#% #//#//!^^"oo    rc Ltdd|d|d|d|d}||S)Nrrerdrf)rrrrerdrfr )rrgptqs rrzGPTQQuantizer.from_configs>}-}-[)l+  4yr) r#r$r%__doc__r rrr!r&rrrs@rrrs:&D$7" 0   rrr~rr)rdrerfrgr?rhc  |td|dt|jdkr%td|dt|jdtj|dk(r td|jd|jd } }|r|dkDr | |zd z |z} nd } | d kDru| |z} | dk7r#|| z } tj |ddgd| ggd }tj ||| |g} tj| d }tj| d }nN|r|d gnd d g}tj ||}tj|d }tj|d }|ritjtj||}tjtj|dtj||}|}tj||}tj|tj|d |}tj|tj |d |}tj"tjtj$d|d |}tj||}|dkDrtj ||}tj&||}tjtj(|dd|}|rd|d z z }d|d z zd z }|rBtj*|tj&tj |d d|z}nPtj tj,tj&tj|||}tj.|||}n}|r?tj*|tj&tj |d d}n 0), the output shapes are [out_features, n_groups] where n_groups is the number of groups along the in_features dimension. Args: x: KerasTensor. The weight tensor to quantize with shape [out_features, in_features]. bits: int. The number of bits to quantize to (e.g., 4). symmetric: bool. Whether to use symmetric quantization. per_channel: bool. Whether to quantize per channel. group_size: int. The group size for quantization. -1 means no grouping. compute_dtype: str. The dtype for computation. Defaults to "float32". epsilon: float. Small value added to (max - min) before computing scale to avoid division by zero. Defaults to 0.0. signed: bool. Whether to use signed quantization range. If True, uses range [-2^(bits-1), 2^(bits-1)-1] (e.g., [-8, 7] for 4-bit). If False, uses range [0, 2^bits-1] (e.g., [0, 15] for 4-bit). Defaults to False. Returns: scale: KerasTensor. The scale tensor for quantization. zero: KerasTensor. The zero tensor for quantization (int8 if signed, uint8 if unsigned). maxq: scalar. The maximum quantization value. z Input tensor z cannot be None.rzInput weight tensor z. must have a rank of at least 2, but got rank rrz!Input tensor 'x' cannot be empty.r*r)constant_valuesrLr~:0yE>)r*r*r"r)r|rrMrsizepadrRrSr5maximumr6rlessnegativeequalrTr4r<powerr3r full_liker9r8tile)rrcrdrerfrgr?rh out_features in_featuresrX remainderrt x_grouped min_values max_valuesreduction_shape x_reshapedmax_abs zero_ranger range_valuesrBrY qmax_signedr zero_dtypes rrlrls X y=+;<== 177|a"1#&%%(\N! 5   xx{a<== ! AGGAJ+LzA~*,q0Z?!|*, >!I-HQFQM2CHAKKL(J#GH WWYQ/ WWYQ/ 1<<,!R[[O4 WWZa0 WWZa0 ++cggj1:>YY HHZ #S\\'%:J  :z2J:s||J'BJOJ:swwz1'=zJJ 88CLL1d!3Q7 GD<< J7L{ww|W5 JJ|T *E IIcnnUA.e XXeW %F%j9JvJ7I $NJ 4 PPrc&tj|d}tj|||}tj|||}tj||j}tjtj |||}|S)a4Rebuild a dequantized weight matrix from group params. This function uses the provided scale and zero tensors to dequantize the input weights_matrix according to the group indices. It maps each position along group_axis of the weights_matrix to its corresponding group parameters and performs the dequantization operation. Args: weights_matrix: Tensor to dequantize. scale: Per-group scale tensor with n_groups along group_axis. zero: Per-group zero-point tensor with n_groups along group_axis. g_idx: 1D integer tensor of length equal to the size of `weights_matrix` along the dimension being dequantized. Each element specifies which group index (0 to n_groups-1) that position belongs to. For example, with 128 columns and group_size=32, g_idx would be `[0,0,...,0, 1,1,...,1, 2,2,...,2, 3,3,...,3]` (32 of each). group_axis: The axis in `scale` and `zero` along which to index using `g_idx`. This determines which dimension of the scale/zero tensors contains the per-group values. Default: -1 (last axis). Returns: A tensor with the same shape as `weights_matrix` containing the dequantized weights produced using the provided group parameters. rArL)rr<rBr/r7rT) rCrBrrDrErF scales_mapped zeros_mapped dequantizeds rdequantize_with_sz_maprNst8XXeW %FHHUFL,, ^\2MK rr)r)rr"r)(rNrnumpyr2 keras.srcrrkeras.src.api_exportrkeras.src.backendrr&keras.src.backend.common.backend_utilsrr keras.src.ops.operationr keras.src.quantizers.gptq_configr rr?rDrJrGrHrxrrrrrrrrrrlr=r?rIrNrrrrVs -)2DM-7& ">?@.M.MA.Mb12  GOO   232jIJ  GOO   7K7t4nC&L01AiA2AH4B=i=*=>  SQ?SQl56 7 <=>89  :  *+~-,~-B,-R.RjEIEX  E3P:$:<$QN%r