mercredi 17 mars 2021

Speeding up Python Lemmatizer with 'for' Loop and 'if' Statements

I use this code to apply a Lemmatizer depending to the postage of a word.

def lemmatize_all(sentence):
    wnl = WordNetLemmatizer()
    lem = []
    for word, tag in pos_tag(word_tokenize(sentence)):
        if tag.startswith("NN"):
            lem.append(wnl.lemmatize(word, pos='n'))
        elif tag.startswith('VB'):
            lem.append(wnl.lemmatize(word, pos='v'))
        elif tag.startswith('JJ'):
            lem.append(wnl.lemmatize(word, pos='a'))
        else:
            lem.append(word)
    return lem

The problem is that the more data I have, the longer it takes. Could you help me to accelerate the code please.

Aucun commentaire:

Enregistrer un commentaire