When it comes to building phrase vectors (by phrase I mean every constituent more than one word up to a sentence), the first thing that comes to mind is to combine the vectors of containing word through some arithmetic operations. This approach has been explored vastly in a series of seminal papers by Mitchell and Lapata ((Mitchell & Lapata, 2008, 2009, 2010).
One simple additive model suggested by (Foltz et al., 1998; Kintsch, 2001; Landauer & Dumais, 1997), the values in input word vectors are summed to form the phrase vector. The logic behind such a model is that if the values in the words vectors show the association of these words with the corresponding context, then the vector of the phrase that is composed of these words should inherit all these contextual features.As an alternative to the additive model, Mitchel and Lapata propose a “componentwise multiplication” model in which input vectors are multiplied component by component.
This way those contextual elements which are not associated with all input words vanish and shared elements are heightened. This model is in line with the intersection view in formal semantics.Some other methods also proposed which more or less can be regarded as weighted variations of additive models. For example, Mitchell and Lapata proposed a weighted additive model in which each input vector is multiplied by a fixed weight. For example, in case of combining subject-verb vector, the subject vector might be multiplied by 0.2 and the verb vector by 0.8, with the intuition that the verb has more weight in depicting the sentence meaning rather than the subject. Other summative weighted models that are worth mentioning are Guevara (2010) and Zanzotto et al.
(2010) where each element is output vector is a weighted sum of all elements of input vectors. A different series of models inspired by formal semantics (Montague, 1970) which is the framework of models proposed by Coecke et al. (2010) and Grefenstette et al. (2011) and look at the composition from the functional approach. Peller and Sadrzadeh (2011a, 2011b) also took the functional approach however from the logical perspective. Clarke (2012) proposed a general mathematical framework for compositionality which includes both functional and non-functional approaches.