Getting to Know NLP

What Is Natural Language Processing

It is the technology by which humans communicate with computers using natural language. It is an interdisciplinary field between linguistics and computer science, and it draws on knowledge from many disciplines, such as mathematics, statistics, linear algebra, probability theory and cognitive psychology.

Mathematics is the core and theoretical guide of all of machine learning, covering probability theory, matrix computation, data analysis, optimization, mathematical statistics, stochastic processes and information theory. There is too much to learn, so it is recommended to first study probability theory and linear algebra, then get a feel for the sub-problems of natural language processing, and gradually learn the related knowledge. For example, language models require some knowledge of probability theory; and if we want to learn sequence labeling, we need to understand some sequence graph models, such as HMM and CRF. Learning in combination with concrete tasks like this lets us learn more efficiently.

Programming practice is the most critical part, which is why people with programming backgrounds make up the largest share of those switching into AI. We need to know the fundamentals of data structures and algorithms, and before interviews we also need to grind through LeetCode interview questions.

Beyond all of the above, we also need to master natural language processing toolkits and machine learning packages, and use deep learning frameworks to build models.

If we have energy to spare while studying, we can also learn more about neuroscience and linguistics. Mastering one more discipline is a very big advantage.

Natural language processing and neuroscience

The convolutional neural network (CNN) originated from a visual experiment on cats, the recurrent neural network (RNN) is closely related to the memory mechanisms in the human brain, and the transformer is an attention mechanism, which is also a field within cognitive science.

Future improvements will certainly be closely tied to cognitive science and neuroscience as well. But on the whole, the human brain is extremely complex, and we have only simulated a small part of it. Some schools of thought say that models that simulate the human brain as closely as possible may be the key path toward strong artificial intelligence.

Linguistics

In concrete tasks, knowing some linguistics is very helpful. For example, for a particular task, whether we use word vectors or character vectors (that is, whether we split the text into units of words or units of characters) depends on the characteristics of the text, the characteristics of the task, and the characteristics of the language itself. Or, when we remove stop words, which parts of speech are likely to be useless involves a fairly deep understanding of parts of speech. Or, whether syntactic analysis is useful for the current task and whether it can reveal the relationships between words: for example, when we build knowledge graphs, the relationships between these words may help us with knowledge extraction. All of this requires some common knowledge of linguistics.

In summary

First master mathematics and computer science

If you have energy to spare, learn about brain neuroscience and linguistics

What is natural language processing from the perspective of artificial intelligence?

NLP is a very central aspect of artificial intelligence. It is a relatively advanced form of intelligence and somewhat harder than image processing. The simplest understanding of artificial intelligence is having certain kinds of intelligence like humans do.

Abilities AI should have ------------------->Applied technologies

1 Computational ability: fast computation and memory storage. Applications: computing speed, storage capacity.

2 Perceptual ability: vision, touch, hearing Applications: speech recognition, synthesis, computer vision, biometrics, images, spatial recognition.

In this area, as with iFlytek’s products, machines can match humans on image processing tasks

3 Cognitive ability: language ability, knowledge management, reasoning and learning

The difference from perceptual ability is that it has an extra intermediate layer. What we see and hear are direct sources of information, but understanding adds a layer of symbols in between, and we need to give these symbols meaning. This is the very hard part.

Applications: natural language processing, which is cognitive intelligence, language interaction, semantic understanding, knowledge learning

Assigning appropriate symbols to perceived things according to different contexts is one of the major challenges natural language processing faces.