Defining "Understanding" through Fine-tuning and Embedding

It sounds like a grand title, but it's not actually that incredible.
It's a memo where I've thought about why "copy-paste engineers" are not ideal and why it's better to be able to read documentation
using more concrete examples.

Example

Programming learning

In the Case of Embedding

Explanation of embedding (from Wikipedia)

"In natural language processing (NLP), a word embedding is a representation of a word. The embedding is used in text analysis. Typically, the representation is a real-valued vector that encodes the meaning of the word in such a way that words that are closer in the vector space are expected to be similar in meaning.[1] Word embeddings can be obtained using language modeling and feature learning techniques, where words or phrases from the vocabulary are mapped to vectors of real numbers."

If we think of search terms as being represented as values in a vector space called Google, and receiving those representations as results:

1. Search for what you're struggling with.
2. Find something that looks relevant from Google.
3. Copy and paste it "as is," or edit it using existing knowledge.

This feels like the method that "copy-paste engineers" tend to use.

In the Case of Fine-Tuning

Explanation of fine-tuning (from Wikipedia)

"In machine learning, fine-tuning is an approach to transfer learning in which the weights of a pre-trained model are trained on new data.[1] Fine-tuning can be done on the entire neural network, or on only a subset of its layers, in which case the layers that are not being fine-tuned are "frozen" (not updated during the backpropagation step)"

to transfer learning in which the weights of a pre-trained model are trained on new data.

If we consider this as strengthening an existing model (concepts, knowledge) for new words or concepts:

1. Read documentation, books, etc., and acquire systematic knowledge.
2. Actually experiment with various things and strengthen the model within yourself.
3. Become able to handle problems in other languages and apply knowledge to new situations.

This feels like the kind of method strong engineers—the type who might release books—would use.

Summary

In practice, I think we usually combine embedding and fine-tuning effectively in our work, but I've explained them here as completely separate concepts for the sake of clarity.

The idea of whether one has "understood" something is a field I'm interested in, and I felt that if I could investigate it more specifically by breaking it down into different methods, it might also be useful for human learning.