ij^8ddlZddlmZddlmZddlmZddlmZddl m Z edGdd eZ ed Gd d eZ y) N)MutableMapping) keras_export)LossScaleOptimizer) Optimizer)serialization_libzkeras.optimizers.OptimizerMapcdeZdZdZd dZedZdZdZdZ dZ d Z d Z d Z ed d Zy) OptimizerMapzA class mapping variables to optimizers. Args: default_optimizer: Default Keras `Optimizer` for any unmapped variables. Nct|tstd||t|tstd|||_|xs t|_y)Nz@default_optimizer must be a Keras Optimizer instance. Received: z.optimizer_map must be a dictionary. Received: ) isinstancer TypeErrordict_default_optimizer_optimizer_map)selfdefault_optimizer optimizer_maps y/var/www/html/emotional.easysim.app/public_html/venv/lib/python3.12/site-packages/keras/src/optimizers/multi_optimizer.py__init__zOptimizerMap.__init__sk+Y7./1   $Z t-L@P #4+5tvc|jSN)rrs rrzOptimizerMap.default_optimizer s&&&rcR||jvr|j|S|jjDcgc]}tj||r|}}t |dkDrt d|dt |dk(r|j|dS|j Scc}w)ahRetrieves the corresponding `Optimizer` by the string key. This method first attempts an exact key match. If no exact match is found, it treats all keys in the map as regular expression patterns and uses `re.fullmatch` to find a policy. For example, to apply a optimizer to all sublayers of an `encoder` block, the key should be explicitly set to `"encoder/.*"`. A key of `"encoder"` will only match the layer with that exact path. Args: key: str. The key to query for an `Optimizer`. Returns: The corresponding `Optimizer`. If no match is found, this method returns `self.default_optimizer`. Raises: ValueError: If the `key` matches more than one regex pattern in the map. Example: ```python >>> from keras.src import optimizers >>> from keras.src.optimizers.multi_optimizer import OptimizerMap >>> opt_adam = optimizers.Adam() >>> opt_sgd = optimizers.SGD() >>> opt_rmsprop = optimizers.RMSprop() >>> optimizer_map = OptimizerMap(default_optimizer=opt_rmsprop) >>> optimizer_map["encoder/layer_0/dense"] = opt_adam >>> optimizer_map["encoder/.*"] = opt_sgd >>> optimizer_map["decoder"] = opt_adam >>> optimizer_map["decoder"].name 'adam' >>> optimizer_map["encoder/layer_0/dense"].name 'adam' >>> optimizer_map["encoder/layer_0/attention/query"].name 'sgd' >>> optimizer_map["decoder/layer_0/dense"].name 'rmsprop' ``` z)Multiple optimizers assigned to variable .r)rkeysre fullmatchlen ValueErrorr)rkeypattern matching_keyss r __getitem__zOptimizerMap.__getitem__$sd $%% %&&s+ + ..335 ||GS)   }  !HQOP P  1 $&&}Q'78 8%%% sB$c`||jvrtd|d|j|dt|tstd|t|tstd|t|t r tdt|t r td||j|<y) zSet optimizer for a given variable. Args: key: string representing the variable path or regex pattern optimizer: Keras Optimizer instance 'z0' already exists in the OptimizerMap with value z.. Please make sure to not use duplicated keys.z key must be a string. Received: z8optimizer must be a Keras Optimizer instance. Received: z'optimizer cannot be LossScaleOptimizer.z#optimizer cannot be MultiOptimizer.N)rr r strrrMultiOptimizer)rr! optimizers r __setitem__zOptimizerMap.__setitem__es $%% %C5,,S123++  #s#?uEF F)Y/&K)  i!3 4FG G i 0BC C#,C rc|j|=yr)r)rr!s r __delitem__zOptimizerMap.__delitem__s    $rc,t|jSr)rrrs r__len__zOptimizerMap.__len__s4&&''rc,t|jSr)iterrrs r__iter__zOptimizerMap.__iter__sD''((rc ||jSr)path)rvariables r__call__zOptimizerMap.__call__sHMM""rctj|jtj|jdS)N)rr)rserialize_keras_objectrrrs r get_configzOptimizerMap.get_configs<!2!I!I''"/EE##   rctj|d|}tj|d|}|||}|S)Nrcustom_objectsr)rdeserialize_keras_object)clsconfigr;rrobj_maps r from_configzOptimizerMap.from_configsN-FF & ' *BB ? #N '7rr)__name__ __module__ __qualname____doc__rpropertyrr$r*r,r.r1r5r8 classmethodr@rrr r sX 6''?&B-4%()# rr zkeras.optimizers.MultiOptimizerceZdZdZdfd ZfdZedZedZedZ e jdZ edZ e jd Z ed Z dd Z d Zd ZdZeddZxZS)r(aAn optimizer wrapper that delegates variables to different optimizers. Initialize the object with an OptimizerMap instance or a callable function that returns an optimizer for a given variable. Example: model.compile( optimizer=MultiOptimizer( OptimizerMap(default_optimizer=optimizers.SGD(), {"encoder/.*": optimizers.Adam()}) ), loss="binary_crossentropy", ) # Or using a callable def optimizer_selector(variable): if "encoder" in variable.path: return optimizers.Adam() else: return optimizers.SGD() model.compile( optimizer=MultiOptimizer(optimizer_selector), loss="binary_crossentropy", ) To access the attributes of the sub-optimizers, iterate over the optimizers using `.optimizers`: For example: optimizer = MultiOptimizer(OptimizerMap( default_optimizer=optimizers.Adam() )) optimizer['.encoder'] = optimizers.SGD() for optim in optimizer.optimizers: print(optim.learning_rate) print(optim.iterations) print(optim.loss_scale_factor) ... The MultiOptimizer class instances will not expose `learning_rate` attribute and will raise an error if accessed. This is because the learning rate might be different for different sub-optimizers. Note: Optimizer-specific callbacks are not supported yet. ct|d||||_g|_t |jdrP|jj D]3}||jvs||_|jj|5t|jdd}|2||jvr#||_|jj|yyy)ac Initialize the MultiOptimizer. Args: optimizer_map: An OptimizerMap instance or a callable function that returns an optimizer for a given variable. loss_scale_factor: It overrides the loss_scale_factor passed to the sub-optimizers. name: The name of the optimizer. g) learning_rateloss_scale_factornamevaluesrN) superrr_inner_optimizershasattrrMrKappendgetattr)rrrKrLopt default_opt __class__s rrzMultiOptimizer.__init__s /  ,!# 4&& 1**113 7d444,=C)**11#6 7 d113FM  #4#9#99,=K )  " " ) )+ 6: $rc|jryi|_i|_tt |j Dcgc]}g}}t |D]"\}}|j|}t|tstd|dt|tr tdt|tr td||j vr=|j|_ |j j||jg|j j|}||j|j!|<||j|j!|<||j|%t#|j |D]\}}|s |j%|t& |I|ycc}w)NzOptimizer for variable z is not an Optimizer instance.z;LossScaleOptimizer cannot be used inside an MultiOptimizer.z7MultiOptimizer cannot be used inside an MultiOptimizer.)built_var_to_optimizer_idx_trainable_variables_indicesrangerrO enumeraterr rr rr(rKrQindex_var_keyzipbuildrN) rvar_list_optimizer_varsivarrSidx variablesrUs rr_zMultiOptimizer.builds :: %'",.)&+C0F0F,G&HI"II) ,FAs%%c*Cc9- -cU3--#12 &#~. M$000(,(>(>%&&--c2%%b)((..s3C=@D & &t}}S'9 :DED - -dmmC.@ A 3  & &s +1 ,6"$"8"8.I %NC )$ %  hCJs F;c|jSr)rOrs r optimizerszMultiOptimizer.optimizerss%%%rc||jdd}|jD]}|j|j|Sr) _variablesrOextendrf)rvarsrSs rrfzMultiOptimizer.variables#s<q!)) 'C KK & ' rctd)NzxLearning rate cannot be accessed on a MultiOptimizer. Access the learning rate on the individual sub-optimizers instead.AttributeErrorrs rrJzMultiOptimizer.learning_rate+s Q  rctd)NzpLearning rate cannot be set on a MultiOptimizer. Set the learning rate on the individual sub-optimizers instead.rn)rvalues rrJzMultiOptimizer.learning_rate2s N  rct|ddS)N_loss_scale_factor)rRrs rrKz MultiOptimizer.loss_scale_factor9st1488rc\||_t|dr|jD] }||_ yy)NrO)rsrPrOrK)rrqrSs rrKz MultiOptimizer.loss_scale_factor=s7"' 4, --- .(-% . .rc|jSr) _iterationsrs r iterationszMultiOptimizer.iterationsDsrct|dk(ry|#|js td|j}|js|j |d|_t|t|j k7r.tdt|dt|j dt t|jDcgc]}g}}t||D]9\}}|j|j|}||j||f;t|j|D]7\}} | s t| \} } |jt| t| 9|jjdycc}w)NrzWhen passing `grads` without `variables`, the optimizer must already be built on a list of variables. Call `optimizer.build(trainable_variables)` first.Tz>Gradients must match trainable variables one-to-one. Received z gradients and z variables.r)rrWr _trainable_variablesr_rYrZrOr^rXr]rQapplylistrv assign_add) rgradstrainable_variablesragrads_and_varsgradrdrerSsub_grads_and_vars sub_gradssub_varss rrzzMultiOptimizer.applyHs u:?   &:: I #'";"; zz JJ* +DJ u:T>>? ?J<t889:+G  ',C0F0F,G&HI"IIU$78 4ID#,,T]]3-?@C 3  & &c{ 3 4 (+  " "N(  ; #C#"&)+=&># 8 $y/4>:  ; ##A&Js Fc:tt|jDcgc]}g}}|D]4}|j|j |}||j |6t |j|D]\}}|s |j|ycc}wr)rZrrOrXr]rQr^finalize_variable_values)rr`rarbrdrerSrfs rrz'MultiOptimizer.finalize_variable_valuesos&+C0F0F,G&HI"II ,C,,T]]3-?@C 3  & &s + ,"$"8"8.I 8NC,,Y7 8 Js BcV|js tdt|j}t |D]#}|j|j ||%|}|j D];}t|j}|dkDs||||z}|j|||z }=y)NzYou are calling `set_weights()` on an optimizer that has not yet been built. Please call `optimizer.build(trainable_variables)` to create the optimizer weights before calling `set_weights()`.r) rWr rrjrZassignrOrf set_weights)rweights own_var_countrcrerS num_opt_vars opt_weightss rrzMultiOptimizer.set_weightsyszzD DOO, }% 2A OOA  % %gaj 1 2)) $Cs}}-La%cC,,>?  ,|#  $rcptj|j|j|jdS)N)rrKrL)rr7rrKrLrs rr8zMultiOptimizer.get_configs5.EE##"&!7!7II   rcl|j}tj|d||d<|di|S)Nrr:rG)copyrr<)r=r>r;s rr@zMultiOptimizer.from_configs:"3"L"L ? #N# }V}r)NNr)rArBrCrDrr_rErhrfrJsetterrKrwrzrrr8rFr@ __classcell__)rUs@rr(r(s0d7B' R&&    99..   %'N8$* rr() rcollections.abcrkeras.src.api_exportr)keras.src.optimizers.loss_scale_optimizerrkeras.src.optimizers.optimizerrkeras.src.savingrr r(rGrrrs` *-H4.-.S>S/Sl/0{Y{1{r