/Expectation-Maximization

(Project of Subject: Machine Learning AUEB 2019) An implementation of Expecation Maximazation algorithm

Primary LanguagePython

Run

Hit the following command to execute the algorithm.

python main.py [OPTIONS]

OPTIONS:
    --segments     INTEGER   The number of segments you want to segment your image (e.g. 4, 8, 16...), default=8
    --path         TEXT      The file path to your image. (e.g. im.jpg), default=im.jpg
    --iterations   INTEGER   The number of iterations you want to run the algorithm, default=100
    --tolerance    FLOAT     The tolerance you accept, default=1e-6

Dependencies

  • matplotlib==3.0.2
  • Click==7.0

Background material in maths

We have the above mixture distribution:


We want to maximize the log Likelihood:


We have to construct the Complete likelihood:


And then the Complete logLikelihood is defined as:


But we don't know the value of the hidden variable z each time, so we must compute the Expected Complete likelihood. Since each appears linearly in the mean value of is obtained by substituting the with their mean value :

where


In order to get the equations of the Maximization Step, we differentiate the above function with respect to and each time and assuming that are constant (in terms of and ) since they have been calculated in the Expectation Step based on the previous parameter values. After that we set the differentiation equals to 0 :




On this step we need to add the LaGrange Multiplier in order to find the exact formula.


And last one:


Results of Experiments - Comparisons with the original Image (Max_steps = 100)


segments=1, Total error: 0.1746


segments=2, Total error: 0.0486


segments=4, Total error: 0.0167


segments=8, Total error: 0.0078


segments=16, Total error: 0.0044


segments=32, Total error: 0.0016


segments=64, Total error: 0.0009