Preliminary considerations about large models

In recent years, the large-scale pre-training language models, represented and originated by BERT and GPT3, have "unexpectedly" become the strategic focus and hotspot of technology competition in the entire artificial intelligence field and leading trends for a period (such as the springing up endlessly Pre-training language models as well as pre-training multi-modal models). NLP based on deep learning is advancing all the way along the track of "extremely large data, large models, and large computing power". This trend has produced a number of surprising but also somehow puzzling phenomena. For example,how should we regard the large model? What related research should be conducted? This report will present some preliminary considerations and discussions on those issues.