Hello and welcome to this lesson on Understanding and Implementing Part-of-speech (POS) Tagging with spaCy!

In this lesson, we'll discuss what POS tagging means, its importance in Natural Language Processing (NLP), and how we can effortlessly perform it using spaCy. By the end of this lesson, you should be able to process a text and tag each token (word) with its corresponding POS using spaCy.

POS tagging is the process of assigning a part-of-speech label (noun, verb, adjective, etc.) to each token (word) in a given text. For example, in the sentence "Sam eats quickly.", "Sam" is a noun, "eats" is a verb, and "quickly" is an adverb. This is important because the meaning of a sentence can significantly be determined by the POS of the words in the sentence.

When we perform POS tagging, it not only identifies the POS of a word, but also its grammatical use within the sentence. For instance, "book" can be a noun ("Sam reads a book.") or a verb ("Book a ticket for me."), and POS tagging helps in distinguishing between these uses.

In NLP tasks like parsing, text-to-speech conversion, machine translation, extraction of relationships and entities, POS tagging plays a crucial role. For example, in information extraction, if you want to extract all named entities that are 'organizations' from some text, knowing that a word is a proper noun (NNP in the detailed Penn Treebank POS tags set) may not be enough; you would need its context among other words in the text.

Implementing POS tagging in spaCy is pretty straightforward. However, it's important to note that POS tagging in spaCy is statistical, meaning it is based on statistical models that consider the context of the words in the text. When we process a text with the nlp object, spaCy tokenizes the text to create a Doc object. This Doc object carries all the computed attributes and properties that we can delve into. For POS tagging, we focus on two token attributes:

• pos_: This is the simple part-of-speech tag, using the Universal POS tag set. It provides a general POS tag, like 'NOUN', 'VERB', 'ADV' etc.
• tag_: This is the detailed part-of-speech tag using the Penn Treebank POS tag set. It provides detailed POS information, like 'VBZ' (verb, 3rd person singular present), 'RB' (adverb), etc.

The power of any learning lies in the doing. Let's roll up our sleeves and dive into some code. For POS tagging with spaCy, we need to process the text and loop through the token properties of the processed Doc object. Let's look at how to do that.

First, we import the spaCy library and load the English language model.

Python
Copy to clipboard
1import spacy
2nlp = spacy.load("en_core_web_sm")

Define a sentence of English text that we want to perform POS tagging on:

Python
Copy to clipboard
1text = "I am learning NLP and using spaCy for POS tagging."

Process this text using the nlp function to create a Doc object:

Python
Copy to clipboard
1doc = nlp(text)

Perform POS tagging on each token in the Doc object using a for loop:

Python
Copy to clipboard
1for token in doc:
2    print(f"{token.text:{10}} {token.lemma_:{10}} {token.pos_:{10}} {token.tag_:{10}}")

This small piece of code will give you the POS tagging information for each word in the text, providing the word (token.text), its base form (token.lemma_), simple POS (token.pos_), and detailed POS tag (token.tag_).

The output of the above code will be:

Plain text
Copy to clipboard
1I          I          PRON       PRP       
2am         be         AUX        VBP       
3learning   learn      VERB       VBG       
4NLP        NLP        PROPN      NNP       
5and        and        CCONJ      CC        
6using      use        VERB       VBG       
7spaCy      spacy      NOUN       NN        
8for        for        ADP        IN        
9POS        POS        PROPN      NNP       
10tagging    tagging    NOUN       NN        
11.          .          PUNCT      .         

This output illustrates the tokenization and POS tagging of a text. Each row represents a token (word or punctuation) from the original text. The columns include the token itself, its lemma (base form), its simple POS tag, and its detailed POS tag. This tagging helps in understanding not just the role of each word in the sentence, but also its base form, which is crucial for many NLP tasks.

What a journey that was! We moved from understanding POS tagging to implementing it using spaCy and working through a practical example. You've now gained a significant NLP skill and you should be able to process a text using spaCy and perform POS tagging on it.

As we transition into the practice tasks, it's important to remember that learning is an iterative process. The more you do, the more you learn and reinforce that learning. The next practice tasks will allow you to apply what you've learnt today and cement this new knowledge. Happy tagging!