Macro Average Precision VS. Micro Average Precision

Note:When the accounts of the returned dcoument lists belonging to different queries are the same, Macro==Micro.
Macro_Micro

Zipf’s law

Zipf’s Law

Four Methods to Estimate Query Language in IR

1.Relevance Model
2.Divergence minimizatioon model
3.simple mixture model
4.regularied mixture model
A Comparative Study of Methods for Estimating Query Language Models with Pseudo Feedback

MEMM v.s. CRF

The advantage of CRF is that CRF resolve the label bias problem which can be happened in the MEMM model by global normalization.
Conditional Random Fields Probabilistic Models for Segmenting and Labeling Sequence Data

The Label Bias Problem

“This per-state normalization of transition scores implies a “conservation of score mass” (Bottou,
1991) whereby all the mass that arrives at a state must be distributed among the possible successor states. An observation can affect which destination states get the mass, but not how much total mass to pass on. This causes a bias toward states with fewer outgoing transitions. In the extreme case, a state with a single outgoing transition effectively ignores the observation. In those cases, unlike in HMMs, Viterbi decoding cannot downgrade a branch based on observations after the branch point, and models with statetransition
structures that have sparsely connected chains of states are not properly handled. The Markovian assumptions
in MEMMs and similar state-conditional models insulate decisions at one state from future decisions in a way
that does not match the actual dependencies between consecutive states.”
Label Bias Problem
Conditional Random Fields Probabilistic Models for Segmenting and Labeling Sequence Data

HMM v.s. MEMM

HMM is a useful model with a long history which has been used in many domains.
MEMM is a new model whih is inspired with HMM and Maximum Entropy theory.This model is more feasible than HMM.It can incorporate many features easily.The HMM also can incorporate features, but the processing is very strange and difficult to operate.
MEMM focuses on p（state|observation, while HMM focuses on p（observation|state）.
Maximum Entropy Markov Models for Information Extraction and Segmentation
hmm-memm-crf

The Maximum Entropy Model

A useful web site:http://homepages.inf.ed.ac.uk/lzhang10/maxent.html
A Maximum Entropy Approach to Natural Language Processing
Lagrangian duality and algorithms for the Lagrangian dual problem