ij4dZddlZddlmZddlmZddlmZddlmZddlm Z ddlm Z dd lm Z dd lm Z d d d dZ d d d dZGddZy)aAWQ (Activation-aware Weight Quantization) algorithm implementation. AWQ protects salient weights by finding optimal per-channel scales based on activation magnitudes, then applies those scales before quantization. Reference: https://arxiv.org/abs/2306.00978 N)ops)Dense) EinsumDense)compute_quantization_parameters)dequantize_with_sz_map)dequantize_with_zero_point)quantize_with_sz_map)quantize_with_zero_pointnum_grid_points group_sizec ftj|d}tj|d}tjtj|dtj ||}d}tj |fd}t|dzD]}||z } | dk(rtj |} ntj|| } tj| d} tjtjtj| tj| } tj| d} tj| | } tj|| } |dk(r1t| d d d dd \} }}t!| | ||}t#|| |}n_t| d d d |d \} }}tjtj$d||zd }t'| | |||}t)|| ||}tj|| }tj*tj,||}tj|tj*|}tj.|}||}| }tj||}|s|}| }|S)aSearch for optimal AWQ scales using grid search. The AWQ algorithm finds scaling factors that protect salient weights. For each channel, we search for an optimal ratio in [0, 1] that minimizes the activation-weighted quantization error. The key insight: we MULTIPLY weights by scales before quantization to expand salient weights. This ensures quantization noise is small relative to the expanded weight magnitude. During inference, we divide by scales to restore the original magnitude. Scale formula: scales = x_max.pow(ratio).clamp(min=1e-4) Loss function: Activation-weighted MSE (approximates output error) Args: weights: Weight tensor [out_features, in_features] (transposed kernel). activation_magnitudes: Per-channel activation magnitudes [in_features]. num_grid_points: Number of grid search points. Defaults to 20. group_size: Group size for quantization (-1 for per-channel). Returns: best_scales: Optimal per-channel scales [in_features]. float32g:0yE>Ndtyperg-C6?r FTbits symmetric per_channelr compute_dtypeint32)rshapecastwhereless ones_likeonesrangepowermaximumsqrtmultiplymaxmindividerr raranger rsquaresubtractmean)weightsactivation_magnitudesrr in_featuresx_max best_loss best_scalesiratioscales scale_meanweights_scaledscale_qzero_qmaxq quantized dequantizedg_idx reconstructed weight_errorweighted_errorloss is_betters m/var/www/html/emotional.easysim.app/public_html/venv/lib/python3.12/site-packages/keras/src/quantizers/awq.pyawq_search_optimal_scalesrEss<))G$Q'K HH*I 6E IIchhud+S]]5-A5 IEI((K>;K?Q& 'L%O# A:]]5)FYYue,FVT*XXcll3776?CGGFOLM [[T2 FJ/gv6  $C ' % !GVT1I5YPK%D %' % !GVTHHSZZ;7:EwOE-I17FEK  ;7 zz#,,w "FG lCJJu4EFxx'  I Ky1I $ YL%\ ctj|d}t||||}tj||}|dk(rQuantize a weight matrix using AWQ. This function performs the complete AWQ quantization process: 1. Find optimal per-channel scales via grid search 2. Apply scales to weights 3. Compute quantization parameters 4. Quantize weights Args: weights_transpose: Weight matrix [out_features, in_features]. activation_magnitudes: Per-channel activation magnitudes [in_features]. num_grid_points: Number of grid search points. group_size: Group size for quantization. Returns: quantized_weights: Quantized weights [out_features, in_features]. scales: Quantization scales [out_features, num_groups]. zeros: Zero points [out_features, num_groups]. awq_scales: AWQ per-channel scales [in_features]. g_idx: Group indices [in_features]. rr r rFTrrrrr) rrrEr&rr zerosrr*r ) weights_transposer/rrr0 awq_scalesr8r9r:r;r<r>s rDawq_quantize_matrixrKs8))-.q1K+' J\\"3Z@NR ? # ! - GVT  ;. :!@ !# ! A{3zA7K) GVUD  * gvz5 88rFc*eZdZdZddZdZdZdZy)AWQaEAWQ quantizer for a single layer. This class accumulates activation statistics during calibration and performs AWQ quantization on layer weights. The AWQ algorithm works by: 1. Collecting per-channel maximum activation magnitudes 2. Using activation magnitudes to determine weight saliency 3. Finding optimal per-channel scales via grid search 4. Applying scales before quantization to protect salient weights Args: layer: The layer to quantize (Dense or EinsumDense). config: AWQConfig instance with quantization parameters. Ncddlm}||_|xs |dd|_d|_t |t s)t |tre|jjdk(rL|jj|_ |jd|_ |jd|_ ||_n!t |tr|jjdk(r|jj|_ t|j}|j!t#|}|dk(r|\}}}||_ ||z|_ n*|dvr|\}}} ||z|_ | |_ nt%d|t'j(t+j,|j|j|jf |_nt/d t1|t+j2|jfd |_y) Nr) AWQConfig)dataset tokenizerr)rrRz9Cannot determine dimensions for EinsumDense kernel shape )kernelz Unsupported layer type for AWQ: rr)keras.src.quantizers.awq_configrOoriginal_layerconfig num_samples isinstancerrrTndimr kernel_shaperowscolumnslayerlistindexr' ValueErrortypesSimpleNamespacerreshape TypeErrortyperHr/) selfr^rWrOrd_model_dim_indexr0headshead_dim out_featuress rD__init__z AWQ.__init__s=#G $$ G  eU # uk *u||/@/@A/E % 2 2D ))!,DI,,Q/DLDJ { + 0A0AQ0F % 2 2D **+E % CJ 7  A%/4, UH' $x/ "f,05-x!H, +  "G% ..{{5<<$))T\\1JKDJ>tE{mLM M&)YY |9%M"rFc| tdtj|dk(r tdt|jdkDr%tj |d|jdf}tj |d}tjtj|d}tj|j||_ |jttj|dz|_ y) aUpdate per-channel activation magnitude statistics. This method tracks the maximum absolute activation value for each input channel across all calibration batches. Args: input_batch: Input activations tensor [batch, ..., in_features]. NzInput tensor cannot be None.rzInput tensor cannot be empty.rRr r)axis) rarsizelenrrdrr'absr$r/rXint)rg input_batchx batch_maxs rDupdate_activation_magnitudesz AWQ.update_activation_magnitudes#s  ;< < 88K A %<= = {  !A %++kB 8I8I"8M3NOK HH[) ,GGCGGAJQ/ &)[[  & & & " ++c#))A,q/.BBrFcddlm}tj|jj }t ||j|jj|jj\}}}}}tj|d}|j|dd\}} } |j`|jjj!||jj"j!||jj$j!||jj&j!||jj(j!|d|j_y)zPerform AWQ quantization on the layer. This method: 1. Runs the AWQ grid search to find optimal scales 2. Quantizes the layer weights 3. Updates the layer's quantized variables r) quantizersr uint8)rnrTN) keras.srcrxr transposer^rTrKr/rWrrr pack_int4rV_kernelquantized_kernelassign kernel_scale kernel_zerorJr>is_awq_calibrated) rgrxweights_matrixr<scalezerorJr>quantized_packed_s rDquantize_layerzAWQ.quantize_layer@s4 )tzz'8'895H   & & KK77{{-- 5 1 5$ EHHY0 ",!5!5 AW"6" !Q    ' ,,334DE ((//6 ''..t4 &&--j9 !!((/04-rFc |`|`y)z"Free memory used by the quantizer.N)r/r^)rgs rDfreezAWQ.freefs  & JrF)N)__name__ __module__ __qualname____doc__rlrvrrrFrDrMrMs +NZC:$5LrFrM)rrbrzrkeras.src.layersrrkeras.src.quantizers.quantizersrrrr r rErKrMrrFrDrsQ "(KBF@D x~ R9jDDrF