B b*@sdZddlmZy ddlZWn(ek rDddlmZedYnXGdddZdd Zd d Z d d Z ddZ ddZ ddZ ddZddZddZd#ddZed krdd!lmZedd"dS)$z8Maximum Entropy code. Uses Improved Iterative Scaling. )reduceN)MissingPythonDependencyErrorz0Install NumPy if you want to use Bio.MaxEntropy.c@seZdZdZddZdS) MaxEntropyaHold information for a Maximum Entropy classifier. Members: classes List of the possible classes of data. alphas List of the weights for each feature. feature_fns List of the feature functions. Car data from example Naive Bayes Classifier example by Eric Meisner November 22, 2003 http://www.inf.u-szeged.hu/~ormandi/teaching >>> from Bio.MaxEntropy import train, classify >>> xcar = [ ... ['Red', 'Sports', 'Domestic'], ... ['Red', 'Sports', 'Domestic'], ... ['Red', 'Sports', 'Domestic'], ... ['Yellow', 'Sports', 'Domestic'], ... ['Yellow', 'Sports', 'Imported'], ... ['Yellow', 'SUV', 'Imported'], ... ['Yellow', 'SUV', 'Imported'], ... ['Yellow', 'SUV', 'Domestic'], ... ['Red', 'SUV', 'Imported'], ... ['Red', 'Sports', 'Imported']] >>> ycar = ['Yes','No','Yes','No','Yes','No','Yes','No','No','Yes'] Requires some rules or features >>> def udf1(ts, cl): ... return ts[0] != 'Red' ... >>> def udf2(ts, cl): ... return ts[1] != 'Sports' ... >>> def udf3(ts, cl): ... return ts[2] != 'Domestic' ... >>> user_functions = [udf1, udf2, udf3] # must be an iterable type >>> xe = train(xcar, ycar, user_functions) >>> for xv, yv in zip(xcar, ycar): ... xc = classify(xe, xv) ... print('Pred: %s gives %s y is %s' % (xv, xc, yv)) ... Pred: ['Red', 'Sports', 'Domestic'] gives No y is Yes Pred: ['Red', 'Sports', 'Domestic'] gives No y is No Pred: ['Red', 'Sports', 'Domestic'] gives No y is Yes Pred: ['Yellow', 'Sports', 'Domestic'] gives No y is No Pred: ['Yellow', 'Sports', 'Imported'] gives No y is Yes Pred: ['Yellow', 'SUV', 'Imported'] gives No y is No Pred: ['Yellow', 'SUV', 'Imported'] gives No y is Yes Pred: ['Yellow', 'SUV', 'Domestic'] gives No y is No Pred: ['Red', 'SUV', 'Imported'] gives No y is No Pred: ['Red', 'Sports', 'Imported'] gives No y is Yes cCsg|_g|_g|_dS)zInitialize the class.N)classesalphas feature_fns)selfr -lib/python3.7/site-packages/Bio/MaxEntropy.py__init__OszMaxEntropy.__init__N)__name__ __module__ __qualname____doc__r r r r r rs4rcCslg}t|jt|jkstxJ|jD]@}d}x,t|j|jD]\}}|||||7}qEvaluate a feature function on every instance of the training set and class (PRIVATE). fn is a callback function that takes two parameters: a training instance and a class. Return a dictionary of (training set index, class index) -> non-zero value. Values of 0 are not stored in the dictionary. r)rr)rxsrvaluesrjfr r r _eval_feature_fnqsr#c sixt|D]\}}||<qWfdd|D}g}t|}xL|D]D} d} x(t|D]} | | | || fd7} qXW|t| |qFW|S)zCalculate the expectation of each function from the data (PRIVATE). This is the constraint for the maximum entropy distribution. Return a list of expectations, parallel to the list of features. csg|] }|qSr r ).0y) class2indexr r sz+_calc_empirical_expects..r) enumeraterrgetrfloat) rysrfeaturesindexkeyZys_iZexpectNfeaturesrr )r&r _calc_empirical_expectss  r2c Csht||||}g}xP|D]H}d}x,|D] \\}} } |||| | 7}q*W||t|qW|S)zCalculate the expectation of each feature from the model (PRIVATE). This is not used in maximum entropy training, but provides a good function for debugging. g)_calc_p_class_given_xitemsrr) rrr,rp_yxZexpectsr0sumrr!r"r r r _calc_model_expectss r7c Cstt|t|f}t|t|ks*txHt||D]:\}}x0|D]$\\}}} ||||| 7<qHWq6Wt|}x0tt|D] } t|| } || | || <qW|S)zCalculate conditional probability P(y|x) (PRIVATE). y is the class and x is an instance from the training set. Return a XSxCLASSES matrix of probabilities. ) numpyzerosrrrr4exprr6) rrr,rprob_yxr0rxr%r"rzr r r r3s   r3cCsNt||f}x:|D]2}x,|D] \\}}}||||7<q"WqW|S)z/Calculate a matrix of f sharp values (PRIVATE).)r8r9r4)r/Znclassesr,f_sharpr0rr!r"r r r _calc_f_sharps  r?cCsd}d}x||krd} } xZ|D]N\\} } } || | | t||| | }| |7} | ||| | 7} q$W|| || |} } | | }||8}t||krP|d}q Wtd|S)z,Solve delta using Newton's method (PRIVATE).grrz Newton's method did not converge)r4r8r:fabs RuntimeError)r/r0r>Z empiricalr;max_newton_iterationsnewton_convergedeltaitersZf_newtonZ df_newtonrr!r"ZprodZratior r r _iis_solve_deltas  & rFc Csht||||}t|} |dd} x@tt|D]0} t| || ||| |||} | | | 7<q0W| S)zBDo one iteration of hill climbing to find better alphas (PRIVATE).N)r3rrrF) rrr,r>r e_empiricalrBrCr5r/Z newalphasrrDr r r _train_iiss rH'h㈵>d绽|=c ss tdtt|kr$td|}} tt|fdd|D} ttt| } t|| | } dgt| } d}x||krt|| | | | ||}ddt| |D}tt j |d}|} t }| ||_ |_ |_|dk r||||krPqWtd |S) aFTrain a maximum entropy classifier, returns MaxEntropy object. Train a maximum entropy classifier on a training set. training_set is a list of observations. results is a list of the class assignments for each observation. feature_fns is a list of the features. These are callback functions that take an observation and class and return a 1 or 0. update_fn is a callback function that is called at each training iteration. It is passed a MaxEntropy object that encapsulates the current state of the training. The maximum number of iterations and the convergence criterion for IIS are given by max_iis_iterations and iis_converge, respectively, while max_newton_iterations and newton_converge are the maximum number of iterations and the convergence criterion for Newton's method. zNo data in the training set.z2training_set and results should be parallel lists.csg|]}t|qSr )r#)r$r)r training_setr r r'+sztrain..grcSsg|]\}}t||qSr )r8r@)r$r<r%r r r r'@sNzIIS did not converge) ValueErrorrsortedsetr?r2rHrrr8addrrrrrA)rMZresultsrZ update_fnZmax_iis_iterationsZ iis_convergerBrCrr+r,r>rGrrEZnalphasZdiffrr )rrMr trains@   rR__main__) run_doctest)verbose)NrIrJrKrL)r functoolsrr8 ImportErrorZBiorrrrr#r2r7r3r?rFrHrRr Z Bio._utilsrTr r r r  s2    =  $ D