Question

What is the difference between parameter estimation using MLE and Bayesian?

When finding the posterior mean, will it be bigger or smaller than the MLE estimate and why?

Answer 1

Difference between MLE and Bayesian estimation will be as:-

Maximum likelihood estimation refers to using a probability model for data and optimizing the joint likelihood function of the observed data over one or more parameters. It's therefore seen that the estimated parameters are most consistent with the observed data relative to any other parameter in the parameter space.

Bayesian estimation is a bit more general because we're not necessarily maximizing the Bayesian analogue of the likelihood. However, the analogous type of estimation (or posterior mode estimation) is seen as maximizing the probability of the posterior parameter conditional upon the data. Usually, Bayes' estimates obtained in such a manner behave nearly exactly like those of ML. The key difference is that Bayes inference allows for an explicit method to incorporate prior information.