Problems and countermeasures in natural language processing evaluation

NLP evaluation is an intuitive manifestation of the current technical level and research progress in NLP field, it also provides a benchmark and direction for the future development of related models and methods. Because of the guiding significance of NLP evaluation, it will seriously affect the fairness and objectivity of work evaluation in related fields if the evaluation itself has deviations.New evaluation data sets and tasks have been continuously proposed in recent years. However, the existing evaluation also exposed a series of problems in terms of scientificity and objectivity. Inappropriate evaluation will limit the advancement of natural language processing technology. This report will analyze the current situation and existing problems of NLP evaluation, as well as propose assumptions and prospects for future NLP evaluation.