General Question

malevolentbutticklish's avatar

How can I obtain a list of normalized English search inputs?

Asked by malevolentbutticklish (2155points) April 25th, 2010

For instance, if someone types ValueJet, AirTran, value jet, airtrain, air train, valeujet, value jet airline, airtran airline, ValuJet, etc these should all map to “AirTran Airways”. I don’t want to call out to a service such as Google. I need to store the information locally so that I can make millions of requests quickly. I am also not trying to perform any sort of Internet search… only to normalize terms so that I could, for instance, count their frequency of occurrence.

Observing members: 0 Composing members: 0

2 Answers

MorenoMelissa1's avatar

I think this may help you, feel free to try it out.

mattbrowne's avatar

What you need is linguistic software handling morphology i.e. inflections, prefixes and endings including all the exceptions e.g. child as the singular of children. Semantic software is even more powerful handling synonyms and related terms. I think this website also offers links to free software.

Answer this question




to answer.

This question is in the General Section. Responses must be helpful and on-topic.

Your answer will be saved while you login or join.

Have a question? Ask Fluther!

What do you know more about?
Knowledge Networking @ Fluther