MEMM v.s. CRF



The advantage of CRF is that CRF resolve the label bias problem which can be happened in the MEMM model by global normalization.
Conditional Random Fields Probabilistic Models for Segmenting and Labeling Sequence Data

Posted in Uncategorized | Leave a comment

The Label Bias Problem

“This per-state normalization of transition scores implies a “conservation of score mass” (Bottou,
1991) whereby all the mass that arrives at a state must be distributed among the possible successor states. An observation can affect which destination states get the mass, but not how much total mass to pass on. This causes a bias toward states with fewer outgoing transitions. In the extreme case, a state with a single outgoing transition effectively ignores the observation. In those cases, unlike in HMMs, Viterbi decoding cannot downgrade a branch based on observations after the branch point, and models with statetransition
structures that have sparsely connected chains of states are not properly handled. The Markovian assumptions
in MEMMs and similar state-conditional models insulate decisions at one state from future decisions in a way
that does not match the actual dependencies between consecutive states.”
Label Bias Problem
Conditional Random Fields Probabilistic Models for Segmenting and Labeling Sequence Data

Posted in Uncategorized | Leave a comment

HMM v.s. MEMM


HMM is a useful model with a long history which has been used in many domains.
MEMM is a new model whih is inspired with HMM and Maximum Entropy theory.This model is more feasible than HMM.It can incorporate many features easily.The HMM also can incorporate features, but the processing is very strange and difficult to operate.
MEMM focuses on p(state|observation, while HMM focuses on p(observation|state).
Maximum Entropy Markov Models for Information Extraction and Segmentation
hmm-memm-crf

Posted in Uncategorized | Leave a comment

The Maximum Entropy Model

A useful web site:http://homepages.inf.ed.ac.uk/lzhang10/maxent.html
A Maximum Entropy Approach to Natural Language Processing
Lagrangian duality and algorithms for the Lagrangian dual problem

Posted in Uncategorized | Leave a comment

The Basic Model of EM Algorithm

The EM algorithm is a general model for maximum-likelihood estimation estimation where the data are “incomplete” or the likelihood function involes latent varibles.
The Basic Model of EM Algorithm
A Note on the Expectation-Maximization (EM) Algorithm

Posted in Uncategorized | Leave a comment

The Lagrange Multiplier Method

The lagrange multiplier method is a useful approach to solve the optimal problem under several limited conditions.
The attachment is in Chinese.
Thanks for the web site of http://www.survivor99.com/entropy/zxw/C12a.htm .
The lagrange Multiplier Method

Posted in Uncategorized | Leave a comment

The Proof of why the KL divergence is not smaller than zero

KL divergence:
sum[p(x)log(p(x)/q(x))]
And,sum[p(x)log(p(x)/q(x))]>=0, please see the attachment for the proof.
The proof of why the KL divergence is not smaller than zero

Posted in Uncategorized | Leave a comment

Publication List

全文检索系统中多关键词查询功能的实现(含封面)全文检索系统中多关键词查询功能的实现
易物模型及其求解算法
人口模型
2008mum
吉林大学代码库

Posted in Uncategorized | Leave a comment

Use the WinRAR Software to Compress/Uncompress Files through Command Line

http://www.freezq.cn/article/455.htm
The above web page is not sufficient to understand how to use this software.
Please refer to the file ‘WinRAR.chm’ for more information which will appear in your installation directory when you install the WinRAR software.

Posted in Uncategorized | Leave a comment

A useful derivate equation

This slideshow requires JavaScript.


3 from《Automatic Query Refinement using Lexical Affinities with Maximal Information Gain》

Posted in Uncategorized | Leave a comment