ijEHddlmZddlmZedGddeZy)) keras_export)QuantizationConfigzkeras.quantizers.AWQConfigc xeZdZdZdddddddded ed ed ed ed ef fdZedZdZ dZ e dZ xZ S) AWQConfiga Configuration class for AWQ (Activation-aware Weight Quantization). AWQ is a post-training quantization method that identifies and protects salient weights based on activation magnitudes. It applies per-channel scaling before quantization to minimize accuracy loss. Methodology: 1. Collects activation statistics from calibration data 2. Identifies salient weight channels based on activation magnitudes 3. Searches for optimal per-channel scaling factors via grid search 4. Applies scaling before quantization to protect important weights References: - Original AWQ paper: "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration" (https://arxiv.org/abs/2306.00978) - Reference implementation: https://github.com/mit-han-lab/llm-awq Args: dataset: The calibration dataset. It can be an iterable that yields strings or pre-tokenized numerical tensors (e.g., a list of strings, a generator, or a NumPy array). This data is used to analyze activation patterns. tokenizer: A tokenizer instance (or a similar callable) that is used to process the `dataset`. weight_bits: The number of bits for weight quantization. AWQ presently only supports 4-bit quantization. Defaults to 4. num_samples: The number of calibration data samples to use from the dataset. Defaults to 128. sequence_length: The sequence length to use for each calibration sample. Defaults to 512. group_size: The size of weight groups to quantize together. A `group_size` of -1 indicates per-channel quantization. Defaults to 128. num_grid_points: The number of grid search points for finding optimal per-channel scales. Higher values may find better scales but take longer. Defaults to 20. quantization_layer_structure: A dictionary defining the model's quantization structure. It should contain: - "pre_block_layers": list of layers to run before the first block (e.g., embedding layer). - "sequential_blocks": list of transformer blocks to quantize sequentially. If not provided, the model must implement `get_quantization_layer_structure`. Example: ```python from keras.quantizers import AWQConfig # Create configuration for 4-bit AWQ quantization config = AWQConfig( dataset=calibration_data, # Your calibration dataset tokenizer=your_tokenizer, # Tokenizer for text data num_samples=128, # Number of calibration samples sequence_length=512, # Sequence length for each sample group_size=128, # Weight grouping for quantization num_grid_points=20, # Grid search points for scale search ) # Apply quantization to your model model.quantize("awq", config=config) ``` iN weight_bits num_samplessequence_length group_sizenum_grid_pointsquantization_layer_structurer r r rrrcLt ||dk7rtd|d|dkr td|dkr td|dks|dk(rtd|d|dkr td ||_||_||_||_||_||_||_ ||_ y) Nrz;AWQ only supports 4-bit quantization. Received weight_bits=.rz'num_samples must be a positive integer.z+sequence_length must be a positive integer.zYInvalid group_size. Supported values are -1 (per-channel) or a positive integer, but got z+num_grid_points must be a positive integer.) super__init__ ValueErrordataset tokenizerr r r rrr) selfrrr r r rrr __class__s t/var/www/html/emotional.easysim.app/public_html/venv/lib/python3.12/site-packages/keras/src/quantizers/awq_config.pyrzAWQConfig.__init__Hs  ! ((3}A7  ! FG G a JK K ?jAo22<Q@  a JK K "&&.$.,H)cy)Nawqrs rmodezAWQConfig.modepsrc:d|jd|jS)zReturns the dtype policy string for this configuration. Returns: A string representing the dtype policy, e.g. "awq/4/128". zawq//)r rr s rdtype_policy_stringzAWQConfig.dtype_policy_stringts# d&&'q(9::rc dd|j|j|j|j|j|j dS)N)rrr r r rrrr r s r get_configzAWQConfig.get_config|sH++++#33//#33,0,M,M  rc|di|S)Nrr)clsconfigs r from_configzAWQConfig.from_configs}V}r)__name__ __module__ __qualname____doc__intdictrpropertyr!r$r& classmethodr* __classcell__)rs@rrrs?L"!-1&I  &I  &I&I&I&I'+&IP;  rrN)keras.src.api_exportr(keras.src.quantizers.quantization_configrrrrrr6s0-G*+F"F,Fr