Big data provides researchers with valuable sources of information for studying demographic behavior in the population. One such source is the texts posted by social network users on various demographic issues. This study utilizes methods for automatically extracting user opinions from the “VKontakte” social network. The extracted texts are then classified using the Conversational RuBERT neural network model to investigate opinions related to reproductive behavior in the population. The classification process addresses two consecutive problems. Firstly, it aims to identify whether a user’s comment contains argumentation. Secondly, if an argument is present, it seeks to determine its type within the context of the “personal-public” dichotomy. To search for arguments and classify their types, six experiments were conducted, varying the dataset and the number of classes. The method employed for automatic extraction and classification of user opinions on the “VKontakte” social network has demonstrated the ability to accurately classify users’ comments, identifying the presence of argumentation and categorizing the arguments within the “personal-public” dichotomy. This enables the identification of personal and social attitudes, values, stories, and opinions, thus facilitating the study of reproductive behavior.

reproductive behavior, personal-public dichotomy, automatic opinion extraction, argumentation, VKontakte, Conversational RuBERT

Big data offers researchers new and valuable sources of information for studying demographic behavior in the population. One of these sources is the statements made by social network users on various demographic issues. In order to examine the reproductive behavior of the population, we initially analyzed this data by determining the “demographic temperature” of two types: the emotional background of specific demographic groups with pronatalist and antinatalist persuasions (Kalabikhina et al. 2021a), as well as the emotional background of statements related to reproductive behavior and the assessment of demographic policies (Kalabikhina et al. 2021b; 2022). We identify positive or negative attitudes towards reproductive aspects such as large families, childlessness, and abortions; however, automatically determining the arguments behind these attitudes remains a challenge.

While natural language processing methods are advancing rapidly, there are still complex problems without straightforward solutions. One such challenge is the identification and reproduction of arguments through automated methods. However, the ability to analyze statements and their justifications is of great interest, particularly in the field of social sciences. Solving this problem would enable us to answer questions not only about what people express in their opinions, but also the underlying basis for those opinions. In this study, we aim to conduct an analysis of argumentation regarding issues related to reproductive behavior based on user comments on the VKontakte social network.

Our study relies on methods for automatically extracting user opinions from the VKontakte social network, followed by the classification of these opinions using the Conversational RuBERT neural network model. The classification of statements serves two purposes: firstly, to identify the presence of argumentation within a user’s comment, and secondly, to determine the type of argument within the context of the “personal-public dichotomy.”

We adopt this approach based on the traditional consideration of the population’s attitudes in the context of demographic behavior to predict behavior and make decisions in the field of demographic policy. For example, we examine attitudes towards the desired number of children individuals plan to have and their perceptions of the ideal number of children in a given society at a specific time.

To address these challenges, we conducted six experiments using different datasets and varying the number of classes (two or three). This paper analyzes the key quality metrics of the trained classification models, including accuracy and F-score, and concludes by selecting the most relevant models for both tasks.

In recent years, researchers have increasingly utilized comments from social media platforms like Reddit and Twitter as a valuable tool for studying public opinion. Social media data is regularly employed to assess the emotional spectrum and sentiment analysis (Thelwall and Stuart 2019; Mittos et al. 2020; Liu et al. 2021; Al-Rawi et al. 2021).

Several studies have focused on extracting arguments from text data for various purposes and research areas, often utilizing machine translation systems. For instance, researchers created an argument-annotated corpus of the Russian language and evaluated the performance of various classifiers for argumentation analysis, which also extended to the analysis of argumentation in COVID-19 datasets (Fishcheva and Kotelnikov 2019; Kotelnikov et al. 2022).

Regarding argumentation extraction, it is worth mentioning studies that have identified comments as a source for extracting arguments. Some authors have conducted such studies using comment corpora from YouTube videos (Sagredos and Nikolova 2022) and news posts (Ehret and Taboada 2020). However, Castellano Parra et al., while analyzing comments on news posts and social media pages of several Spanish newspapers, found that both the level of engagement (number of comments per reader) and the proportion of reasoned comments in social networks were higher compared to news portals (Castellano Parra et al. 2020). Additionally, researchers note that on both platforms, arguments and viewpoints tend to belong to a limited circle of active users, which can contribute to the monopolization of discourse by certain individuals and groups (Jensen 2016).

The analysis of arguments from social media comments has gained popularity in the research community, particularly due to the increased discussions surrounding COVID-19-related topics such as vaccination and quarantine (Melton et al. 2021; Wawrzuta et al. 2021, 2022; Karami and Anderson 2020). Furthermore, comments are utilized for monitoring and making operational decisions regarding COVID-19 and its consequences (Han et al. 2020; Oyebode et al. 2021).

There are individual studies that focus on analyzing comments expressing attitudes towards abortion and its legalization (Hasan and Ng 2013; Graells-Garrido et al. 2019; Misra et al. 2017). It is also worth highlighting works that analyze comment content on other demography-related topics, including parenthood aspects (Mencarini et al. 2017), health problems (Shah et al. 2019), the impact of various factors on demographic processes such as natural disasters (Mandel et al. 2012) and the COVID-19 pandemic, including vaccination for children (Miao et al. 2020; Glandt et al. 2021; Liu and Liu 2021; Thorpe Huerta et al. 2021; Abosedra et al. 2021), sexual harassment or violence (Andalibi et al. 2016; Xue et al. 2019; Al-Rawi et al. 2021; Lin et al. 2022), and attitudes towards genetic tests (Mittos et al. 2020), among others.

The analysis of social network comments within the context of demographic topics is not yet widely explored in the Russian academic literature. However, the use of user-generated content in demographic research, in general, has been increasing in recent years. There have been attempts to analyze the sentiment of Russian-language comments on social networks to determine the “demographic temperature” within pro-natalist and anti-natalist groups of users (Kalabikhina et al. 2021a). Furthermore, studies have focused on analyzing the opinions of social network users regarding reproductive behavior based on comments on VKontakte (Kalabikhina et al. 2021b; 2022). It has been demonstrated that comments can be a better source for analyzing the mood of statements compared to posts (Kalabikhina et al. 2021a).

Additionally, several datasets consisting of comments by pronatalists and antinatalists have been published, which serve as the basis for research data (Kalabikhina and Banin 2020; 2021).

Other researchers primarily concentrate on assessing the emotional background of posts (Donchenko et al. 2017) and comments (Sidorov and Slastnikov 2021; Smetanin and Komarov 2021), without specifically extracting the arguments underlying users’ positions.

Indeed, the automated extraction of arguments from comments and determining their nature is a challenging task. Automating the entire process of extracting arguments, especially in the context of demographic topics, is highly complex. However, manually processing texts after automatically selecting comments with arguments can be time-consuming. As a result, the initial step towards automating argument extraction has involved identifying broader classes of arguments that can be applied to various demographic topics.

In this study, arguments are classified based on the “personal-public” dichotomy, depending on whether the commentator relies on personal experiences or references public ideas as supporting evidence. This approach is rooted in the division of opinions regarding individual and ideal aspects of reproductive behavior. For example, when assessing reproductive attitudes, questions often involve the expected and ideal number of children. The expected number refers to the children the respondent personally intends to have, considering their individual circumstances. On the other hand, the ideal number represents the respondent’s perception of the societal norm at that given time. Sometimes, questions may address the desired number of children, which is close to a normative question but focuses on individual plans irrespective of specific circumstances.

This approach is relevant for studying reproductive behavior and people’s attitudes towards demographic and family policies. By distinguishing between personal arguments/stories and statements about social norms, rules of demographic behavior, or motivations behind others’ actions, it becomes possible to monitor the balance between “personal” and “public” perspectives. Monitoring the increasing share of “personal” arguments can provide valuable insights and help address concerns related to demographic topics. Filtering personal arguments allows for a more focused analysis and identification of specific arguments within the dataset.

It is important to note that the division between personal and public arguments is not limited to reproductive issues alone. Similar divisions can be observed in various domains, such as marital decisions, migration, or self-preservation behaviors, where personal and social experiences shape individuals’ attitudes, relationships, and actions. For instance, arguments related to vaccination, including personal reasons against vaccination and discussions about vaccination in society, can also be classified within this framework.

A similar approach was found in the article by Kiesel et al. (2022). The authors aimed to automate the search for values by utilizing sets of values from various opinion polls such as Schwartz and the World Values Survey. The primary objective of their work was to establish connections between people’s statements (in natural language) and their value orientations, and to automate this process by training artificial intelligence to make these connections. It is noteworthy for our study that the authors’ described system of values included a distinction between personal and public values.

In our study, we utilize two sets of demographic data. The primary dataset consists of comments extracted from VKontakte, a social network, focusing on topics related to childbirth (Kalabikhina and Banin 2020; Kalabikhina and Banin 2021). The additional dataset comprises statements made by VKontakte users discussing various aspects of the coronavirus infection (Kotelnikov et al. 2022).

In this study, two classification tasks were conducted: one to determine the presence² of an argument and another to identify the type of argument (public or personal).

A total of six experiments were performed, each addressing a specific problem. Experiments 1-3 focused on determining the presence of an argument, while Experiments 4-6 aimed to classify the type of argument. The data used in Experiments 2 and 3 included both reproductive and COVID-19 data, while the remaining experiments utilized only reproductive data. The training, testing, and validation samples were divided in a ratio of approximately 80:10:10 to ensure an even distribution across the main topics of interest (as described in the “Data collection” section).

The empirical part of the study was conducted using the Python environment with the PyTorch, Transformers, and Scikit Learn libraries. The Conversational RuBERT model³ served as the basis for all experiments. This model is a pre-trained Russian language model that has been additionally trained on social network data and user dialogues. The experiments employed specific parameters, including a learning rate of 0.0005 and a batch size of 64. The number of epochs, ranging from 4 to 6, was selected based on the best F-score obtained on the validation set. The model’s performance was evaluated using accuracy and F1-score metrics, and the results are summarized in Tables 8 and 9. More detailed information on the model’s quality can be found in the Appendix, specifically Tables A1–A6.

In addition to identifying the presence of arguments, our main interest was to determine the content of arguments, specifically the distinction between personal and public arguments. This classification task was important for our study, even at a high level of generalization, to ensure that the model’s performance was not sensitive to specific argument topics, which could affect its overall quality.

As described above, we classified arguments based on the “personal-public” context, i.e., whether the commentator used personal experiences or cited public ideas as an argument.

This task was more challenging than identifying the presence of an argument due to the great number of controversial cases where statements contained elements of both types, but annotators should have chosen only one class to assign to each comment.

To address this, we conducted two types of experiments. In Experiments 4 and 5, we focused on classifying comments with argumentation into two classes. The reproductive data used for training was reduced to 2,049 comments, including only those where at least two out of three annotators indicated the presence of an argument. In Experiment 5, Class “1” represented comments with personal arguments, as agreed upon by at least two annotators, while Class “0” included all other comments, including controversial cases. The ratio of observations between Class “0” and Class “1” was approximately 65:35. Similarly, Experiment 5 classified comments into public-type argumentation, with a ratio of observations between Class “0” and Class “1” at approximately 45:55.

In Experiment 6, we trained the model on the full set of reproductive data, which was divided into three classes: “0” for no argumentation, “1” for public-type argumentation, and “2” for personal-type argumentation. The number of comments in each class was approximately 70:15:15.

The classification results showed that Experiments 1 and 2 achieved the best performance in identifying the presence of an argument. This can be attributed to Experiment 1 training the model exclusively on reproductive data, which reduced the risk of misclassification due to topic differences between reproductive behavior and the pandemic. Moreover, Experiment 2 had a larger number of observations in the training set, which likely contributed to its higher quality compared to other experiments.

In terms of determining the type of argument on the comparison of the quality metric, Experiments 4 and 5 performed better than Experiment 6. This outcome was expected as Experiments 4 and 5 addressed a simpler problem with two classes instead of three. Additionally, these experiments benefited from more reliable data since the reduced sample automatically excluded many ambiguous cases. The distribution of observations across classes in Experiments 4 and 5 was also more balanced compared to Experiment 6.

The annotation of user comments related to reproductive behavior in the VKontakte social network revealed that approximately 40% of the selected comments contained an argument expressing the author’s positive or negative attitude towards the demographic topic. Among these comments, around 40% were classified as personal arguments, while the remaining were classified as public arguments based on social attitudes, values, and norms. These findings indicate that there is a solid foundation for analyzing the arguments related to demographic viewpoints in social network user comments and identifying personal and public arguments.

The method employed in this study, which involved automatically extracting user opinions from VKontakte comments and classifying them using the Conversational RuBERT neural network model, demonstrated its effectiveness in accurately identifying the presence of arguments and determining their types in terms of the personal-public dichotomy.

Throughout the study, six experiments were conducted, varying the dataset and the number of classes, to address the challenges of argument retrieval and classification. The results obtained indicate that the developed model achieved a high level of accuracy in classifying user comments. This automated approach can be further utilized for data analysis in similar contexts. It has the potential to monitor social networks and promptly identify an increase in personal-type critical statements regarding demographic issues, contributing to the improvement of pronatalist socio-demographic policies.

Moreover, the study opens up new possibilities for applying the developed algorithm to diagnose other types of demographic behaviors, such as self-preservation, matrimonial, and migratory behaviors. Testing the algorithm in the analysis of these behaviors can help identify the scale of personal-type statements and track the dynamics of such statements over time.

In conclusion, the findings of this study support the effectiveness and accuracy of the developed model in classifying user comments. The proposed approach can enable automation in analyzing similar types of data, paving the way for future research and applications in the field of demographic behavior analysis (self-preservation, matrimonial or migratory).

The study was carried out within the framework of research work «The reproduction of the population in socio-economic development».

The work of Natalia Loukachevitch on the COVID-2019 data and their use in the current reproductive behavior study was supported by the Russian Science Foundation (grant No. 21-71-30003).

Kalabikhina Irina – Hab.Doctor (Economics), Professor, Head of Population Department, Faculty of Economics, Lomonosov Moscow State University, Moscow, 119991, Russia. E-mail: kalabikhina@econ.msu.ru

Zubova Ekaterina – Fox Fellow at Yale University, New Haven, CT, 06511, USA. E-mail: ekaterina.zubova@yale.edu

Loukachevitch Nataila – Hab.Doctor (Technical Sciences), leading researcher of Research Computing Center of Lomonosov Moscow State University. Moscow, 119899, Russia. E-mail: louk_nat@mail.ru

Kolotusha Anthony – Candidate of Economic Sciences (Ph.D.), 2nd category Programmer, Faculty of Economics,, Lomonosov Moscow State University, Moscow, 119991, Russia. E-mail: koav13a@econ.msu.ru

Kazbekova Zarina – researcher of Population Department and Digital Economy Research Laboratory, Faculty of Economics, Lomonosov Moscow State University, Moscow, 119991, Russia. E-mail: kazbekova@econ.msu.ru

Banin Evgeniy – Candidate of Technical Sciences (Ph.D.), researcher of Kurchatov Institute, Moscow, 123182, Russia. E-mail: evg.banin@gmail.com

Klimenko German – postgraduate student, Faculty of Economics, Lomonosov Moscow State University, Moscow, 119991, Russia. E-mail: german89000@mail.ru