Wittgenstein 5. Thinking

Today I would like to get back to the topic I have touched already in my first article about Wittgenstein – about the relation between the language and thinking. In his Tractat, Wittgenstein defines the truthfulness of a statement as follows.

He says that there are only three possible outcomes: true statement, lie, or meaningless statement. For a statement to be true, it has to be possible in the world as we know it and verifiable as true. For a statement to be a lie, it has to be possible in the world as we know it and verifiable as false. For the statement to be meaningless, it has to be not possible in the world as we know it. I simplify but I don’t think I lose the essence.

I liked it immediately, probably because it matched the training of my first profession (physics) that required to verify the statement’s truthfulness using an experiment. Besides, each of us acts similarly all the time. We verify the truthfulness of a statement by comparing it to the practice, even when we do this verification in our mind only. Somebody says something, and we think, “how true is that? Have I experienced the same outcome in the same circumstances?”

Problem is that we do not always have the corresponding experience and cannot immediately to make an experiment. In addition, not everything in life can be verified by experiment. So, this way to differentiate truth from lie and meaninglessness is not practical. That’s why more often than not everybody retains their convictions unchallenged. 

Such a conviction and the worldview in general (the world model in the mind) are typically constructed so that the person looks decently in his own eyes – in accordance with the system of values of the social group he belongs to. That is why – without practical verification – it is very difficult (if possible at all) to convince a person that his view is not correct. And the harder we push, the more powerful defense he mounts.  

It was noted that we tend to agree with those, whom we like. And why do we like somebody? We can talk about it for days. Meanwhile, we make our minds about liking a person in a few first seconds. The con men exploit this our tendency to their monetary advantage all the time.

But let us get back to Wittgenstein. He seems to state that the mind is a social phenomenon, not private. And he considers the language and thinking tightly coupled if not outright the same. In my view, language is definitely the way to involve others in problem-solving, thus making it possible to tackle more complex problems. We also use language for passing knowledge to future generations, thus securing the progress we observe since the time humans acquired first speech, then writing, and especially after the printing press was invented and now – the internet. 

From this point of view, I can agree with Wittgenstein that the mind of each of us is a part of the joined mind of the society we live in. We cannot avoid influence from our environment, the books we are reading, and the sites we are visiting.

According to Wikipedia, thought encompasses an “aim-oriented flow of ideas and associations that can lead to a reality-oriented conclusion.” I don’t see a reason to disagree.

In his Philosophical Investigations, Wittgenstein states that the meaning of a word or a sentence is defined by its usage. It seems that we acquire knowledge the same way – from considering the specific facts and then making a generalization. When we hear a story or just a statement, we compare it with our experience or just trust the speaker if we feel like this.

We are often forced to make decisions when there is not enough reliable information. That is how we surround ourselves with myths.

After the USSR collapsed, many “truths” of the past turned out to be myths. Any country has its own mythology. Very rarely and not everybody has a chance to realize it. When it happens though, the process of getting rid of the myths can be rather painful. That’s why we avoid it, if not forced or motivated by the quest for truth.

But I have deviated from the topic. The language we use while “talking” to a computer does not allow such freedom. There is no trust or not trust. The instructions are provided and the computer executes them. What would happen if the computer doubted the validity of our instructions?

Meanwhile, a human, even in doubt, finds a way to make a decision, may be based just on the liking the instructor or not. Or on using some kind of moral principles. Or just from sheer compassion. Such irrational decision making apparently was beneficial, since human species managed to survive so far. This means that our “gut feeling” is not a bad judge. Human emotions and systems of automatic decision making were honed for millions of years of trials and errors. The best approaches were selected by evolution. The computers do not have an evolutionary history. So, they stuck in the realm of specific facts and have just to execute commands, without thinking.

That’s how it was until recently – before new approaches were developed, namely machine learning, neural networks, deep learning, and even evolutionary programming – those are the buzz words of our time. all of them are related to artificial intelligence development. It is a huge rapidly progressing area of research. I will even not try to cover all of it. I will stick only to one thread, along which I happened to work as a programmer.

Let’s start at the top and move down to the most interesting details.  Artificial intelligence includes several approaches:  statistical methods, traditional symbolic modeling, and computational intelligence (CI). We will look into the last one.

CI has an advantage over the traditional binary approach (based on 0/1). Human language does not fit well into such a simplistic model. We just have discussed that the meaning of word or sentence depends on the context and on what the speaker means actually would like to communicate. It also depends on his motivation and world view as a whole.

CI uses a different logic, called fuzzy logic. It better fits for molding human language and thinking, because it allows ascribing to each element not just 0 or 1, but the degree of closeness to 0 or 1. This way we can create overlapping sets of descriptions, pretty much as we do it in our language.

CI includes five directions, approaches, or principles (the terminology is still under construction): fuzzy logic, neural networks, evolutionary programming, learning theory, and probabilistic methods. 

A substantial subset of CI is called machine learning, and the subset of machine learning – based on artificial neural networks – is called deep learning. Neural networks allow the computer to learn possible solutions using examples. There is no need to incorporate in the programming code the rules specific for each of the problems. The algorithm constructs and invents the rules as needed. For example, if we collect a good number of images and mark some of them “cat” and others “not a cat”, the algorithm will be able to learn how to identify a cat among other – new – images. Such an approach is called supervised learning. A human acts as a teacher in this case, who helps the student (computer) to learn human language.

But much more interesting and promising unsupervised learning, when an algorithm classifies the images (for example) on its own, without using a hint from a “teacher.” Naturally, the computer, in this case, does not call an image of a cat “a cat.” It “invents” its own language. Only after the classification is complete, this new language can be translated into human one by naming the categories (if such categories do exist in human language). In this case, a human learns from the computer.

Now, do you remember our starting point about computers just executing the instructions? Using unsupervised learning, the computer decides itself what to do. It even may ask a human to translate the results from the computer language to the human one. Who will be executing whose instructions in this case?

So far, as I have written in the previous article, it is just mapping between the computer language the objects of the real world. It is very similar to Wittgenstein’s approach in  Tractatus: “[human] language models real world and reflects its structure; this way the meaning of a word/sentence comes from the real world.” I added the square brackets around “human” in order to imagine what Wittgenstein would say about not so human language.

It is easy now to imagine how a computer can start introducing more nuanced classifications, “watching” the moods of the humans around it, for example. Will it be able to learn to understand the emotions?  I don’t see why not. But then will it be able to start “experience” them – similarly to how the children first imitate the adults? Will we be able to discern their imitation from the “real experience”?

At this point, we can talk about such things only theoretically. Deep learning rapidly grows in complexity and hits the limits of the level of problems it can tackle.  It seems that evolutionary programming can take over and work on the problems that are beyond the deep learning capacity of comparable complexity. It is very different from neural networks. Its goal is to generate computer code (a program) specific to each problem. Such a program is going to be less complex as it is not a general-purpose program, will work faster, and require fewer resources.

Today the most popular algorithm of evolutionary programming is called a genetic algorithm. It uses mechanisms similar to natural selection. People, who work in this area use the terms “mutation”, “selection”, “crossover”. 

Sounds familiar? Remember how – just a few paragraphs above – we talked that  “human emotions and systems of automatic decision making were honed for millions of years of trials and errors. The best approaches were selected by evolution. The computers do not have an evolutionary history. So, they stuck in the realm of specific facts and have just to execute commands, without thinking.” What can we say now?