Application of Time-Weighted PageRank Method with Citation Intensity for Assessing the Recent Publication Productivity and Partners Selection in R&D Collaboration

Abstract

This article considers the problem of assessing the recent publication productivity of scientists based on PageRank class methods and proposes to use these assessments to solve the problem of selecting scientific partners for R&D projects. The methods of PageRank, Time-Weighted PageRank, and the Time-Weighted PageRank method with Citation Intensity (TWPR-CI) were used as a basis for calculating the publication productivity of individual subjects or scientists. For verification, we used the Citation Network Dataset (Ver. 14) of more than 5 million STEM publications with 36 million citations. The dataset is based on data from ACM, DBLP, and Microsoft Academic Graph databases. Only those individual subjects who published at least two articles after 2000, with at least one of these articles cited at least once before 2023 year, were analyzed. Thus, the number of individual subjects was reduced to 1,042,122, and the number of scientific publications was reduced to 2,422,326. For each of the methods, a range of estimates of productivity is indicated, which are obtained as a result and possible options for making decisions on the selection of potential individual subjects as performers of R&D projects. One of the key advantages of the TWPR-CI method is that it gives priority to those researchers who have recently published and been cited frequently in their respective research areas. This ensures that the best potential R&D project executors are selected, which should minimize the impact of subjective factors on this choice. We believe that the proposed concept for selecting potential R&D project partners could help to reduce the risks associated with these projects and facilitate the involvement of the most suitable specialists in the relevant area of knowledge.

1. Introduction

In the creation of research and development (R&D) projects, joint scientific research is an important component of the development of innovation and technologies, the expansion of scientific directions, and the creation of interdisciplinary research communities. This is the key to scientific communication and mobility, which affects the technological and social development of the state. The task of forming a team for R&D projects or selecting scientific partners to perform tasks within the framework of R&D projects is difficult. To achieve the goals of an R&D project, it is necessary to select such scientists who would be leaders in their industries. In addition, most projects have an interdisciplinary focus, especially for large R&D projects, for which a consortium of universities and other institutions and organizations should be formed. In such projects, the performers involve specialists from different industries to perform different work packages. For example, these are packages related to project management and governing, quality management, the promotion of project results, etc. Often, the choice of partners for R&D projects is carried out subjectively by the project manager or is formed based on the range of common scientific connections of the project manager and other participants. This approach is irrational for the formation of a project team. After all, the close scientific connection of the project participants can lead to the goals and objectives of the project being achieved locally within the framework of their joint experience and developments. To obtain innovative and scientifically significant results from R&D projects, it is crucial to find new solutions, and for this, the experience of a wide range of narrow specialists in a specific field of knowledge is important. In particular, ref. [1] illustrates that private enterprises can enhance their scientific and technological capabilities by collaborating with partners, including researchers, to implement scientific projects. It might be beneficial for private enterprises to consider expanding beyond their current boundaries in order to enhance their capabilities [2]. This is linked to the growing complexity and diversity of knowledge in the technological sphere, as well as the social and economic challenges that we are facing today [3]. There is a wealth of research dedicated to the concept of open innovation, to methods and strategies for developing R&D projects, and to collaboration between private companies and higher education institutions [4,5]. It would seem that insufficient attention has been paid to the development of methods for identifying optimal scientific partners for the creation of productive R&D projects. It would seem that the existing methodology for partner selection does not fully take into account the criteria for selection, perhaps due to a lack of in-depth analysis. The algorithms used for partner selection do not always reflect the realities of the selection process and are constrained by mathematical limitations [3].

In the current business environment, it is becoming increasingly common for companies to collaborate with various partners, including private companies and universities from different countries, in order to realize their project activities [6,7]. It seems reasonable to suggest that such collaborative arrangements have the potential to amplify the impact of research and development alliances on corporate innovation thanks to the synergistic effects that can be generated as a result [8,9]. Ref. [10] suggests that the ability of companies to engage in effective collaboration may increase over time, particularly in the context of a growing number of such partnerships, including those with academic partners. Ref. [11] suggests that collaboration with more partners within the context of R&D projects could potentially lead to greater synergy. It would be beneficial to consider the potential for synergies and reciprocal knowledge transfer across disciplinary boundaries, which could facilitate the production and sale of innovative products.

One distinctive aspect of selecting scientific partners for R&D projects is the incorporation of a criterion that considers their publication productivity over the past few years, with consideration given to the specific field of knowledge. It could be suggested that the publication productivity of a scientist is a certain quantitative assessment, determined by the number of publications and the quality of those publications. The term “quality of publication” is understood to encompass the number of citations, the intensity of those citations, and their relative importance compared to other publications in the same field, based on citation network analysis.

The conceptual foundation of forming an R&D project team based on a recent publication productivity assessment is an attempt to evaluate the ex ante impact of the preceding factors. It is thought that a preliminary evaluation of potential scientific partners and their objective selection process will positively affect the overall effectiveness and quality of the project. In refs. [12,13], the conceptual foundation for ex ante impact assessment is presented, where proposals are considered as promises of achievement. It would be beneficial to consider which aspects of academic knowledge could be evaluated in relation to societal needs and the coherence of measures within a project. Ref. [14] offers insight into the organization of distributed project teams. Given the advances in information technology and the current social context, it is becoming increasingly important to collaborate on R&D projects. In such a case, the possibility of a rigid attachment between selected scientific partners and a specific city or country is excluded. This creates an opportunity to conduct comprehensive research on potential partners regardless of their country of residence. It is particularly important to consider this when implementing R&D projects, as leading specialists in specific narrow fields may be located in different parts of the world.

The application of analytical tools, network citation, and scientific collaboration for the selection of research partners is associated with a number of challenges. One of the most significant challenges arises from the fact that such networks accumulate information about citations and connections between researchers over an extended period of time. It would be advisable to consider only their recent scientific activity and publication record when selecting potential research partners. Given the importance of selecting researchers who have been actively engaged in scientific work over the past few years, it seems prudent to give priority to those who have made significant contributions to research and development projects. It is thought that such a selection could potentially improve the project’s starting position and reduce the risk. One of the main criteria for selecting scientific partners is the assessment of their publication productivity. For this reason, in order to address the issue of the ageing of scientific research, one method of assessing the recent publication productivity of scientists was selected from the group. Similarly, PageRank takes into account the time of the publication of articles, weighs the assessments, and considers the intensity of citations.

It might be suggested that one issue that arises in the choice of scientific partners for R&D projects is the decision of the project manager, which is based on one’s own perspective. In this study, an attempt is made to move away from the subjective nature of the choice of scientific partners and to develop a quantitative selection method based on the analysis of the citation network of scientific publications of potential partners. At the same time, it is important to consider the need for inter-organizational cooperation. Ref. [15] shows that in order to create new ideas, it is important to support a high degree of diversity of project participants. At the same time, trust and dedication, which, in this context, are often considered key to the successful implementation of projects, are really not important. In [16], it is also stated that the choice of the optimal strategy for the development of the project and the achievement of results is determined by cooperation with heterogeneous partners.

It should also be noted that in the context of globalization and high competition in the innovation market, enterprises face fierce competition in foreign markets. The solution to the issue of the competitiveness of such enterprises lies in the realm of ensuring reliable cooperation and project activity with scientific institutions and organizations. Such cooperation makes it possible to increase technological capabilities and contributes to increased competitiveness and profit growth. For example, ref. [17] analyzes the specifics of choosing partners for R&D cooperation by small- and medium-sized enterprises in Korea. It is concluded that enterprises willingly cooperate with scientific institutions and universities if they produce an innovative product, which is difficult for competitors to repeat. In [18], it is indicated that the cooperation of small- and medium-sized enterprises within the framework of R&D projects is the key to improving their economic efficiency and can ensure the sustainable formation of technological standards for product quality. In [19], R&D cooperation in Germany was analyzed, and it was concluded that such cooperation is beneficial for enterprises and improves their innovative capabilities regardless of the type of cooperation. Thus, ensuring cooperation in the framework of scientific and innovative projects between enterprises and scientific institutions, as well as between scientific institutions, is relevant for the development of all participants in such cooperation. These articles do not describe the methods that make it possible to form a project team based on evidence-based recommendations without the influence of subjective factors.

For a rational choice of partners in the framework of R&D projects, the structure of projects and the needs of projects should be considered to effectively achieve goals and objectives. Ref. [20] analyzes the peculiarities of uniting non-profit universities and other partners into alliances. It has been concluded that such alliances, for the most part, increase the productivity of scientific research. In [21], depending on the structure of the project, indices of the activities of potential partners for their selection into the project are analyzed. It is indicated that the established structure is an additional tool for making decisions on the organization of R&D projects. However, an in-depth analysis of the publication activity of potential project participants is not considered in this article. Indeed, to effectively select project participants, it is important to analyze the achievements of each potential participant in the relevant field of knowledge. The participation of such performers may affect the value of the results of R&D projects. That is why, to select potential performers from the scientific industry, their publication productivity should be analyzed and evaluated.

This can be achieved, for example, based on the analysis of the indices of publication activity of scientists: h-index [22], g-index, i10-index, etc. However, such methods have disadvantages, since when calculating index estimates, part of the information about the citation of authors that is outside the core of the assessment is lost. Therefore, for a comprehensive analysis of the publication productivity of potential project partners, it is necessary to analyze the citation network comprehensively, considering all citations and published articles. Special consideration should be given to those articles that have recently been published because the selected partner should be actively involved in the scientific research process and seeking to publish their results now, not many years ago. In addition, it is important that the intensity of citations of a potential partner’s publications is sufficient at the current time. This means that the author’s published research receives attention and is potentially significant for this field of knowledge.

To account for all articles and citations of potential R&D project partners, you can use a citation network analysis based on the PageRank (PR) method. In the classical view, this method is used to calculate the impact of web pages on social networks [23,24]. However, the principle of considering all links in the citation network without exception made it possible to use modifications of this method to assess the reputation or publication productivity of individual scientists or authors of articles, as well as teams of scientists from higher education institutions in which they work. The disadvantage of the PR method for assessing publication productivity is that it is ineffective to use it to evaluate new higher institutions with an insignificant publication history [25]. Also, obviously, the method will not be effective for assessing the productivity of scientists with a minor publication history, since they have started active scientific activity in the last 5–10 years, for example. In the case of choosing scientific partners to carry out R&D projects, the prospectus for a short period of time—that is, the history of publication activity in the last few years—is very important. After all, scientists who are actively published and cited in a specific field of knowledge should be selected. Scientists who have been published and cited in the past but are now inactive should not have an advantage in selection.

Ref. [26] considers a modification of the PR method that takes into account the age of publication. As a result, it is shown that this modification allows for greater efficiency compared to the conventional PR method. In [27], it is indicated that the analysis of the citation network with weighting factors that consider the aging of the publication can provide valuable information about the current state of the authors’ publication activity. Accordingly, this can help to make a rational decision on the choice of partners for joint R&D projects. The weighted PR (WPR) method is described in [28]. Ref. [29] describes the constructed time-weighted PR method (TWPR). However, in addition to weighing by time, it is important to consider the intensity of citations of scientific publications of scientists. The higher the intensity of quoting a scientist’s publications, the more priority should be given to its selection as part of R&D projects in a specific field of knowledge. That is, the assessment of the scientist’s productivity, which is the selection criterion, should be higher. Ref. [25] uses the Time-Weighted PageRank method with Citation Intensity (TWPR-CI), which has shown its effectiveness for assessing the productivity of higher education institutions and scientific organizations, including new ones founded in the last 20 years. An urgent task is to explore the possibilities of using this method to assess the productivity of scientists and solve the problem of their selection by potential performers in R&D projects.

The aim of this study is to assess the recent publication productivity of researchers based on the Time-Weighted PageRank method with Citation Intensity and to use these estimates to solve the problem of selecting research partners for R&D projects. The advantage of this method is the high efficiency of assessing the productivity of new higher education institutions. The method can be adapted to find the estimates of publication productivity not of higher education institutions but of individual researchers working in these institutions within the relevant research area. The method has high sensitivity for assessing new research actors, including new higher education institutions and researchers who have recently started their careers. This allows for an efficient calculation of the recent publication productivity of researchers, which is important for ensuring a rational choice of partners for R&D projects.

The objectives of the study are as follows: (1) to describe the Time-Weighted PageRank method with Citation Intensity for assessing the publication productivity of researchers within a specific research area; (2) to verify this method to solve the problem of selecting research partners for R&D projects, taking into account the priority research area of scientists and the needs of R&D project management.

2. Methodology and Data

2.1. Basic Terms and Concepts

A number of terms and concepts are used in this article. Let us describe the main ones. An individual subject in the context of scientific research is a scientist who personally publishes articles and is actively engaged in scientific activity within a certain research area. An individual entity is affiliated with one or more collective entities, that is, institutions of higher education, scientific organizations whose products are scientific developments, and innovations. Scientific activity in this work is considered through the prism of publication activity. Accordingly, the links between the citation of scientific publications of individual or collective subjects are constructed in the form of a citation network. Certain scientific articles are associated with relevant individual subjects who are the authors of the articles. In addition, since each author is associated with a certain collective subject, the articles, respectively, are also associated with higher education institutions, faculties, and departments in which the authors work. Each individual subject works in a specific research area, determined by the subject of the author’s research and the publication activity. The concept of a research area can have different degrees of detail. For example, one can analyze scientists within the research area “mathematics”, and one can detail separately “mathematical physics”, “differential equations”, “optimization theory”, etc. Finally, the effective selection of individual subjects to the performers of R&D projects depends on this. The productivity of the project directly depends on the experience and scientific productivity of its developers. The productivity of scientific activity in this article is a certain value calculated based on quantitative scientometric indicators based on publication activity, regardless of subjective factors. Productivity can concern both individual subjects, both personally and collectively, and departments, universities, scientific institutes, etc. The publication productivity of a researcher is a certain numerical assessment determined based on the number of publications attributed to that researcher and the quality of these publications. The quality of a publication means the number and intensity of citations, as well as its importance in comparison with other publications in a given research area, and in particular, based on the methods of citation network analysis.

In this article, to assess the publication productivity of researchers, the class of PageRank methods is chosen as a basis, which allows us to calculate the importance of a researcher’s publications in comparison with others in the citation network. The priority in the formation of the assessing of publications by the classical PageRank method is works that were published a long time ago and have a sufficient citation history. This drawback significantly hinders the use of this method in assessing the publication productivity of researchers who can be selected as executors of R&D projects. Therefore, for this task, it is proposed to modify PageRank and take into account the time of publication and citations, as well as the intensity of citations.

The more intensively a publication is cited over a short period of time, the more interesting it is to the scientific community. Individuals who have been cited more intensively over a recent period of time should be prioritized for selection to a research team. After all, this indicates that they have been actively engaged in scientific activity in recent years. Mathematically, the value of citation intensity can be defined as the angle coefficient of the time series of the number of citations for a certain period of time relative to the abscissa axis. To take into account the intensity of citations in addition to the age of citations in the assessment of the publication productivity of individual subjects, the Time-Weighted PageRank with Citation Intensity method can be used. This method has been validated for assessing the productivity of collective entities, including newly created ones [25]. It can be assumed that the method should work well for assessing the publication productivity of individual subjects within their research area. Such a modification will allow us to qualitatively take into account in the assessment of publication productivity not only well-known scientists with a long publication activity but also scientists who have recently started their careers in the relevant research area. Summing up the above, we obtain a conceptual model for selecting research partners for R&D projects based on the assessment of the publication productivity of researchers using the Time-Weighted PageRank method with Citation Intensity (Figure 1).

The input to this model is the citation network. These data are used to assess the publication productivity of researchers. The Time-Weighted PageRank method with Citation Intensity combines the advantages of the Time-Weighted PageRank methods with the citation publication time to increase the publication productivity of those researchers who have published recently. It also takes into account citation intensity to further increase the publication productivity scores of those researchers whose work is currently being cited. Taking into account the distribution of authors’ publications by research area, the model generates output data in the form of a ranked list of the publication productivity estimates of researchers who have publications in the relevant research area. This list is used to select research partners for R&D projects.

2.2. Collection of Data on Citations of Scientific Publications of Individual Subjects or Scientists

To verify this study, an analysis of the dataset on the citation of scientific publications of the Citation Network Dataset (version 14) was carried out [30]. The analysis and peculiarities of dataset formation are described in [31]. This dataset includes data on 5,259,858 scientific publications and 36,630,661 citations of these publications. The data were collected based on the databases DBLP [32], ACM [33], and Microsoft Academic Graph [34]. The specified dataset version encompasses updated citation data for scientific publications as of January 2023. The dataset broadly covers scientific publications spanning the period from 1815 to 2023. However, the temporal distribution of these publications is uneven, with approximately 87% of the collected scientific works having been published between 2000 and 2023.

Based on the unique identifiers of the authors of scientific publications, 2,390,072 individual subjects were identified, among which 2,174,815 had at least one publication citation and participated in the assessing of their scientific results.

When determining the research area to which individual subjects belong, the field of study (FoS) indicator was used as a basis. More detailed information about the methodology for determining this indicator is described in ref. [35]. It should be noted that the FoS is defined only for 3,586,117 publications. Accordingly, based on the FoS indicator, 122,731 research areas have been identified for 2,365,473 individual entities with a high degree of detail that can be considered as potential partners or performers of R&D projects. For this task, such detail is necessary because, as a rule, the tasks of the project relate to certain rather narrow branches of knowledge for which it is necessary to select potential performers-experts in these industries. The results of the distribution of individual subjects between the research area are given in the Supplemental Materials Table S1. The JSON file contains a dictionary, where the identifier of the author or individual subject is the key, and the coefficient is the calculated value. The study of the distribution of authors according to the article was not carried out in this study. However, ref. [36] shows the results of an analysis of the impact of scientific publications depending on the gender composition of their authors. The previous version of this dataset was also used as a basis. This can help if there are specific criteria for selecting R&D project participants, particularly in terms of gender composition.

According to the analysis, most of the scientific publications in the dataset belong to the following scientific fields: computer science, mathematics, machine learning, mathematical optimization, and computer vision, etc. This is a certain limitation, as the humanities area remains unexplored in this article. Nevertheless, this dataset was chosen because the large amount of data allows us to obtain more reliable results. After all, this dataset reflects the publication activity of scientists in STEM over a long period of time, especially over the past 20 years. This fits into the logic of the study to select potential R&D project implementers based on their recent publication productivity.

2.3. Mathematical Description of the Problem

Let us define a set of individual subjects or researchers affiliated with collective subjects or higher education institutions. For each individual entity, an information space can be formed, the components of which are its publication activity. Moreover, each publication belongs to a specific research area ${\mathrm{o}}_{\mathrm{i}}$; $\mathrm{i}=\overline{1,\mathrm{m}}$. Then, for each individual subject, based on the research area of his or her scientific publications, it is possible to determine the individual sets of research areas that correspond to it. Moreover, each individual subject, as an author of articles, may have articles that belong to different research areas, which are cited and displayed in the citation network accordingly.

The citation of one publication in another can be specified through the Markov matrix (A5).

The publication productivity of individual subjects will be determined on the basis of the publication productivity of the research area in which these individual subjects work. Let the coefficient corresponding to the productivity of research area ${\mathrm{o}}_{\mathrm{i}}$ at the q-th step at time ${\mathrm{T}}^{\mathrm{k}}$ be denoted by ${\mathrm{b}}_{\mathrm{i}}^{\mathrm{q},\mathrm{k}}$. According to the PageRank method, at the first step for q = 0, the coefficients for all scientific publications at time ${\mathrm{T}}^{\mathrm{k}}$ are equal and are defined as ${\mathrm{b}}_{\mathrm{i}}^{0,\mathrm{k}}={\displaystyle \frac{1}{{\mathrm{R}}_{\mathrm{k}}^{\mathrm{i}}}}$, $\mathrm{i}=\overline{1,\mathrm{m}}$; ${\mathrm{R}}_{\mathrm{k}}^{\mathrm{i}}$ is the number of publications belonging to research area ${\mathrm{o}}_{\mathrm{i}}$ at time ${\mathrm{T}}^{\mathrm{k}}$; ${\mathrm{R}}_{\mathrm{k}}^{\mathrm{i}}=\mathrm{c}\mathrm{a}\mathrm{r}\mathrm{d}\left({\mathrm{p}}_{\mathrm{w}}^{\mathrm{j},\mathrm{k}}\right)$, if $\Lambda \left({\mathrm{p}}_{\mathrm{w}}^{\mathrm{j},\mathrm{k}},{\mathrm{o}}_{\mathrm{i}}\right)=1$ All other coefficients will be calculated iteratively using Formula (A6).

Taking into account the age and citation intensity of the TWPR-CI method, it is necessary to group individual subjects by research area. In this way, it will be possible to immediately determine the weight of each individual subject in the relevant research area, taking into account the publication productivity of other individual subjects in this research area. Then, the initial value of the coefficient will be calculated using Formula (A7).

The formula of the TWPR-CI method introduces a linear aging coefficient for the publications of an individual subject within the relevant research area. It should be noted that the closer the time of publication to the end point of time ${\mathrm{T}}^{\mathrm{N}}$, the greater the impact of this publication. Accordingly, in this case, a higher assessment of the productivity of an individual subject within the relevant research area is taken into account. Scientific publications that were published a long time ago will have less influence on the formation of an individual’s publication productivity score. If $\mathsf{\beta }=0$, then the TWPR-CI method is transformed into the TWPR method without taking into account the citation intensity. If ${\mathrm{b}}_{\mathrm{i}}^{0,\mathrm{k}}={\displaystyle \frac{1}{{\mathrm{R}}_{\mathrm{k}}^{\mathrm{i}}}}$, then the method is transformed into the conventional PageRank. The arctan function allows us to calculate the citation intensity of scientific publications of an individual subject as the angular coefficient of the straight line drawn between the number of citations of scientific publications of an individual subject at a given time ${\mathrm{T}}^{\mathrm{k}}$ and the number of citations of publications at a given time ${\mathrm{T}}^{\mathrm{N}}$. As a result, we obtain a vector estimate of the productivity of the research area ${\mathrm{o}}_{\mathrm{i}}$, $\mathrm{i}=\overline{1,\mathrm{m}}$. To calculate the productivity of each individual subject ${\mathrm{v}}_{\mathrm{j}}$ in the relevant research area ${\mathrm{o}}_{\mathrm{i}}$, we normalize the scores of the individual subject’s belonging to the relevant research area ${\mathrm{o}}_{\mathrm{i}}$ via the method of minimum–maximum scaling. In this way, we calculate the contribution of each individual subject ${\mathrm{v}}_{\mathrm{j}}$ to the productivity of the research area ${\mathrm{o}}_{\mathrm{i}}$, which is an estimate of the publication productivity of the individual subject ${\mathrm{v}}_{\mathrm{j}}$. Once the ranked lists of potential project implementers are formed, the task of appointing them is solved and the work of the relevant department is organized to coordinate their participation and functional responsibilities. For a more detailed description of the method, see Appendix A.

3. Results

Using the Citation Network Dataset (v.14), binary mappings between scientific publications and individual subjects are computed. A citation graph is then constructed, representing the citation relationships between scientific publications across different subjects. This citation graph is a directed, weighted graph, where the weight of each arc corresponds to the number of citations between scientific publications.

To solve the problem of selecting the performers of R&D projects, it is necessary to select those who have been actively engaged in scientific activities during the last period. Accordingly, the distribution of the years of the first and last publication of individual subjects for this dataset was constructed. Figure 2 shows the number of personal subjects by the years when they published their first publication on this dataset [30]. The number of individual subjects by year when they published their last publication on the dataset [30] is shown in Figure 3. Figure 4 shows the difference between the last and first year of publication for each of the individual subjects. As can be seen from Figure 4, the number is inversely proportional to the length of this period, and it is impossible to single out certain individual subjects who may not meet the selection requirements for the performers of R&D projects. Therefore, the publication activity of all individual subjects can be analyzed for the purpose of joining projects. It should be noted that to analyze the difference between the time of the last and the first publication of individual subjects, data on their publications since 1970 were selected. Prior to this period, the number of articles in the dataset was small and time-limited since 1815, so it is not displayed in the figures.

Based on the constructed citation graph, the publication productivity of individual subjects is calculated using the PR (A6), TWPR (A6), and (A7) methods for $\mathsf{\beta }=0$и TWPR-CI (A6), and (A7) for $\mathsf{\beta }={\displaystyle \frac{1}{2}}$ (see Appendix A). The value of the citation intensities of scientific publications is also calculated in the first part of the Formula (A7). Estimates of publication productivity are found based on the iterative method with accuracy $\mathsf{\epsilon }={10}^{-5}$.

Subsequently, the estimates of publication productivity were normalized relative to the maximum value. As a result, all estimates of the publication productivity of collective subjects fall within the [0, 1] range. Only publications by authors who were published between 2000 and 2024 were selected for the analysis. The choice of such a period is explained by the uneven distribution of the number of publications by city in the Citation Network Dataset (ver. 14). In addition, for the task of choosing partners, the analysis of the publication productivity of individual subjects over the last period is prioritized, which is relatively close to the time of the formation of the project team. It should also be noted that individual subjects who published only one article based on the results of the dataset analysis were excluded from consideration. This is since such individual entities cannot be on the list of potential project performers due to low publication activity. In addition, the convergence of methods from such a dataset correction are significantly accelerated; that is, individual subjects who published at least two articles after 2000, with at least one of these articles being cited at least once before 2023, were chosen as the basis for the analysis. Thus, the number of individual subjects was reduced to 1,042,122, and the number of scientific publications was reduced to 2,422,326.

Figure 5 shows a histogram of the distribution of the producer–news ratings of individual subjects using the PR method. As can be seen in Figure 5, the estimates are grouped around the indicator 0.9, which may indicate a close relationship between the publications listed in the Citation Network Dataset (ver. 14). Very few publications have the highest rating of 1.0. When using the PR method to select potential performers of R&D projects, a threshold of about 0.9 can be set. All individual subjects whose PR productivity ratings exceed the specified threshold will be selected as potential project performers.

Figure 6 shows a histogram of the distribution of the producer–news ratings of individual subjects using the TWPR method. Apparently, the time weighting has significantly changed the appearance of the chart. Currently, the majority of personal subjects have productivity in the range of [0, 0.4]. Considering the needs of the project, a threshold coefficient can also be set in this case. For example, all individual subjects with a TWPR productivity score of more than 0.4 are potential performers. Individual subjects can also be sorted by the maximum productivity score.

Next, the intensity of citations of articles by individual subjects was calculated. Figure 7 shows that a small portion of the articles has a maximum citation intensity index. The closer this indicator is to 1, the higher the score in terms of the intensity of citations of scientific publications. The histogram of the distribution of productivity ratings of individual subjects based on the calculated citation intensity index, considering time weighting (TWPR-CI), is shown in Figure 8. As you can see, the estimates are charged to zero. On a logarithmic scale, these estimates have a more visual representation (Figure 9).

The estimates obtained using the TWPR-CI method can be ranked, and a list of potential project performers can be formed, considering the assessment of publication productivity and the intensity of citations of their scientific publications over the last period.

For example, consider the task of finding project partners or project executors, referred to as “scientific collaboration”. To do this, we will calculate the contribution of each individual subject to productivity using the TWPR-CI method with Formula (A7) (see Appendix A). Considering the found values of the partner’s publication activity belonging to a given research area, we arrange, in descending order, the productivity of each individual subject. As a result, we obtain 105 individual subjects, 10 of which with the highest scores are shown in Table 1, $\mathsf{\beta }={\displaystyle \frac{1}{2}}$. The names of scientists are anonymized by identifiers from the Citation Network Dataset. Considering the results of the calculation, the project manager can contact these potential performers and discuss their possible participation in the project.

As can be seen from Figure 10, most assessments are concentrated near zero; however, there are a certain number of individual subjects who receive the maximum productivity assessment in a specific field of knowledge or research area. It should be noted that in addition to one priority research area, from which it is necessary to choose a potential project contractor, there may be several such research areas for a particular project. Then, the estimates can be weighted by priority relative to the importance of a particular area of knowledge for the project.

4. Discussion

4.1. Findings

As a result, the application of the methods PR (A6), TWPR (A6), and (A7) for $\mathsf{\beta }=0$ and TWPR-CI (A6), as well as (A7) for $\mathsf{\beta }=0.5$ and $\mathsf{\lambda }=0.1$ (see Appendix A), was described for the task of selecting scientific partners or potential performers of R&D projects. For the analysis, individual subjects who published at least two articles after 2000 were selected if at least one of these articles was cited at least once before 2023. This allows us to immediately reject authors who are not cited or who have published only one publication, which obviously does not qualify you for the list of potential project performers.

To solve the problem, we employed the Citation Network Dataset (ver. 14), and the presence of duplicates and errors was checked. Estimates of the productivity of individual subjects were calculated using the specified methods. For each of the methods, the range of productivity estimates obtained as a result, as well as the possible options for making decisions on the choice of potential individual actors as project performers, are indicated. To do this, we can employ a threshold coefficient to immediately cut off obviously unacceptable candidates. Alternatively, we can arrange the estimates and, as a result, output the names of potential performers whose estimates of publication productivity are maximum. It should be noted that estimates of publication productivity using the PR method are charged up to the value of 0.9. Most estimates using the TWPR method are placed on the segment [0, 0.4]. The scores, according to the TWPR-CI method, are raised to 0, and the maximum scores according to this method are obtained by the minimum number of individual subjects. The advantage of the TWPR-CI method is that when calculating the estimates, priority is given to individual subjects who have been intensively cited and published in the relevant research area over the most recent period. This is what ensures the rationality of choosing the best potential performers of R&D projects.

Comparing the results of applying the PR (Figure 5), TWPR (Figure 6), and TWPR-CI (Figure 8) methods to assess the productivity of individual subjects, it was shown that the PR method can produce anomalous distributions of estimates that may prevent the correct selection of partners for a joint R&D project. Ref. [25] showed that the TWPR-CI method has the advantage of being more sensitive to the publication productivity estimates for new collective subjects on average during the first ten years of observation. If we compare the results of the proposed method of selecting partners based on productivity assessment with the Bayesian network method [37], we can see that it has the advantage of giving higher scores to scientists with recent scientific publications. Instead, the use of the Bayesian model allows us to take into account the non-technical skills of the candidates

It should be noted that to perform the tasks of a certain R&D project, complex specialists may be required, i.e., those who are experts in several fields of knowledge, for example, physics and differential equations or chemistry and nanotechnology, etc. In this case, the estimates of individual subjects for all research areas are summarized with the corresponding weight coefficients that determine the priority of a specific research area for the tasks of the project.

The obtained results for the verification of the TWPR-CI method allow us to conclude the effectiveness of its use for the task of selecting scientific partners for R&D projects.

4.2. Limitation and Future Research Lines

One of the limitations that obviously determines certain estimates of publication productivity is the structure and features of the Citation Network Dataset (ver. 14), which was analyzed in this paper. In particular, based on the productivity assessment using the PR method, it can be assumed that the relationship between scientific publications due to citations is high. This may affect the productivity results obtained via other modified methods. However, conceptually, the algorithm for calculating estimates using the TWPR and TWPR-CI method does not change from this. It should also be noted that in this study, the distribution of publications to the relevant research area was carried out based on FoS data in the Citation Network Dataset (ver. 14). It is possible that the use of other methods of distributing scientific publications to specific research areas may affect the results of the assessment.

Another feature of this dataset is that most of the articles relate to the field of mathematics, information technology, artificial intelligence, software engineering, etc. It can be assumed that the distribution of individual subjects according to estimates of publication productivity for the humanities may differ. But this is a separate task for future research. Another separate task is to study the optimal values of the parameters and to calculate the estimate using the TWPR-CI method.

The effectiveness of the proposed method for selecting potential partners for R&D projects requires further research. In particular, the change in the Salton index [38] can be used for the study.

5. Conclusions

This study presents the results of the application of PR (A6), TWPR (A6), and (A7) for $\mathsf{\beta }=0$, and those of TWPR-CI (A6) and (A7) for $\mathsf{\beta }=0.5$ for the task ($\mathsf{\lambda }=0.1$) of calculating the intensity of publication productivity and selecting partners or performers of R&D projects. To verify this method, a dataset of scientific publications and citations was selected, namely, the Citation Network Dataset (ver. 14), which is publicly available. The dataset contains publications for the period from 1815 to 2023. In addition, individual subjects who published at least two articles after 2000, with at least one of these articles being cited at least once before 2023, were selected as the basis for this analysis. Thus, the number of individual subjects was reduced to 1,042,122, and the number of scientific publications was reduced to 2,422,326. For each method, a range of productivity estimates and decision-making options for selecting potential individual subjects as performers of R&D projects were calculated. Based on the TWPR-CI method, it is possible to calculate the publication productivity of individual subjects, considering the intensity of citation of their publications during the period of time immediately preceding the decision to form a project team. This feature allows us to select project partners or project executors which rationally correspond to the research area in question. Such a choice, which maximally rejects the influence of subjective factors on the formation of a project team, can basis the selection process towards high productivity and innovative results.

One of the key advantages of the TWPR-CI method is that it gives priority to those researchers who have recently published and been cited frequently in their respective research areas. This ensures that the best potential R&D project executors are selected, which should minimize the impact of subjective factors on this choice. We believe that the proposed concept for selecting potential R&D project partners could help to reduce the risks associated with these projects and facilitate the involvement of the most suitable specialists in the relevant area of knowledge.

Author Contributions

Conceptualization and methodology, O.K. and Y.A.; software, Y.A. and S.T.; analysis, O.K., Y.A., A.M. and A.F.; coding, Y.A.; writing—original draft preparation, O.K. and Y.A.; writing—review and editing, A.B., A.M., A.F. and K.K.; visualization O.K. and. Y.A.; project administration, A.B. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

All data are available in this publication. The publicly available datasets analyzed in this study can be found here: Citation Network Dataset: DBLP+Citation, ACM Citation network, Aminer, available online via the following link: https://www.aminer.org/citation (accessed on 10 August 2024).

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Table

Table 1

Estimates of the contribution of individual subjects to productivity in the area of “scientific collaboration” are calculated using the TWPR-CI method.

№	ID	Estimate
1	5631388845cedb3399d142e5	1.000
2	5631388845cedb3399d142e9	1.000
3	53f4547fdabfaeecd69e4272	0.716
4	6325d2da293b827d85782e25	0.601
5	53f44ea8dabfaee4dc7f26dc	0.589
6	5630e39c45cedb3399bc8a99	0.511
7	5630e39c45cedb3399bc8aa5	0.511
8	5630e39c45cedb3399bc8a9d	0.417
9	53f45e71dabfaedf43628333	0.393
10	562c43c245cedb3398bd4288	0.370

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/publications12040048/s1, Table S1: Author research area based on Citation Network Dataset FoS.

References

1. Fontana, R.; Geuna, A.; Matt, M. Factors affecting university–industry R&D projects: The importance of searching, screening and signalling. Res. Policy; 2006; 35, pp. 309-323. [DOI: https://dx.doi.org/10.1016/j.respol.2005.12.001]

2. Hung, K.P.; Chiang, Y.H. Open innovation proclivity, entrepreneurial orientation, and perceived firm performance. Int. J. Technol. Manag.; 2010; 52, pp. 257-274.

3. Yoon, B.; Song, B. A systematic approach of partner selection for open innovation. Ind. Manag. Data Syst.; 2014; 114, pp. 1068-1093.

4. West, J.; Bogers, M. Leveraging external sources of innovation: A review of research on open innovation. J. Prod. Innov. Manag.; 2013; 31, pp. 814-831. [DOI: https://dx.doi.org/10.1111/jpim.12125]

5. Erden, Z.; Von Krogh, G.; Kim, S. Knowledge sharing in an online community of volunteers: The role of community munificence. Eur. Manag. Rev.; 2012; 9, pp. 213-227.

6. Nieto, M.J.; Santamaria, L. The importance of diverse collaborative networks for the novelty of product innovation. Technovation; 2007; 27, pp. 367-377.

7. Belitski, M.; Delgado-Márquez, B.L.; Pedauga, L.E. Your innovation or mine? The effects of partner diversity on product and process innovation. J. Prod. Innov. Manag.; 2024; 41, pp. 112-137. [DOI: https://dx.doi.org/10.1111/jpim.12696]

8. Belderbos, R.; Carree, M.; Diederen, B.; Lokshin, B.; Veugelers, R. Heterogeneity in R&D cooperation strategies. Int. J. Ind. Organ.; 2004; 22, pp. 1237-1263.

9. Lavie, D.; Miller, S.R. Alliance portfolio internationalization and firm performance. Organ. Sci.; 2008; 19, pp. 623-646.

10. Heimeriks, K.; Duysters, G. Alliance capability as a mediator between experience and alliance performance: An empirical investigation into the alliance capability development process. J. Manag. Stud.; 2007; 44, pp. 25-48. [DOI: https://dx.doi.org/10.1111/j.1467-6486.2006.00639.x]

11. Belderbos, R.; Carree, M.; Lokshin, B. Complementarity in R&D cooperation strategies. Rev. Ind. Organ.; 2006; 28, pp. 401-426.

12. Benneworth, P.; Olmos-Peñuela, J. An openness framework for ex ante evaluation of societal impact of research. Res. Eval.; 2024; 33, rvac023. [DOI: https://dx.doi.org/10.1093/reseval/rvac023]

13. Twyman, M.; Newman, D.A.; DeChurch, L.; Contractor, N. Teammate invitation networks: The roles of recommender systems and prior collaboration in team assembly. Soc. Netw.; 2022; 68, pp. 84-96. [DOI: https://dx.doi.org/10.1016/j.socnet.2021.04.008]

14. Morrison-Smith, S.; Ruiz, J. Challenges and barriers in virtual teams: A literature review. SN Appl. Sci.; 2020; 2, 1096. [DOI: https://dx.doi.org/10.1007/s42452-020-2801-5]

15. Gattringen, R.; Wiener, M.; Strehl, F. The challenge of partner selection in collaborative foresight projects. Technol. Forecast. Soc. Change; 2017; 120, pp. 298-310.

16. Heger, T.; Boman, M. Networked foresight—The case of EIT ICT Labs. Technol. Forecast. Soc. Change; 2015; 101, pp. 147-164. [DOI: https://dx.doi.org/10.1016/j.techfore.2014.02.002]

17. Hong, S.H. Determinants of Selection of R&D Cooperation Partners: Insights from Korea. Sustainability; 2021; 13, 9637. [DOI: https://dx.doi.org/10.3390/su13179637]

18. Jiang, H.; Gao, S.; Song, Y.; Sheng, K.; Amaratunga, G. An empirical study on the impact of collaborative R & D networks on enterprise innovation performance based on the mediating effect of technology standard setting. Sustainability; 2019; 11, 7249. [DOI: https://dx.doi.org/10.3390/su11247249]

19. Shapiro, C.; Varian, H.R. Information Rules: A Strategic Guide to the Network Economy; Harvard Business School Press: Boston, MA, USA, 1999.

20. Choi, J.; Contractor, F.J. Improving the Progress of Research & Development (R&D) Projects by Selecting an Optimal Alliance Structure and Partner Type. Br. J. Manag.; 2017; 30, pp. 791-809.

21. Geum, Y.; Lee, S.; Yoon, B.; Park, Y. Identifying and evaluating strategic partners for collaborative R&D: Index-based approach using patents and publications. Technovation; 2013; 33, pp. 211-224.

22. Hirsch, J.E. An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. USA; 2005; 102, pp. 16569-16572.

23. Leskovec, J.; Rajaraman, A.; Ullman, J.D. Mining of Massive Datasets; Stanford University Press: Palo Alto, CA, USA, 2014; 513.

24. Brin, S.; Page, L. The anatomy of a large-scale hypertextual Web search engine. Comput. Net. ISDN Syst.; 1998; 30, pp. 107-117.

25. Kuchansky, A.; Biloshchytskyi, A.; Andrashko, Y.; Biloshchytska, S.; Faizullin, A. The Scientific Productivity of Collective Subjects Based on the Time-Weighted PageRank Method with Citation Intensity. Publications; 2022; 10, 40. [DOI: https://dx.doi.org/10.3390/publications10040040]

26. Wang, Y.; Zeng, A.; Fan, Y.; Di, Z. Ranking scientific publications considering the aging characteristics of citations. Scientometrics; 2019; 120, pp. 155-166.

27. Fiala, D.; Tutoky, G.; Koncz, P.; Paralič, J. Ageing of Edges in Collaboration Networks and its Effect on Author Rankings. Acta Polytech. Hung.; 2015; 12, pp. 149-160.

28. Xing, W.; Ghorbani, A. Weighted PageRank algorithm. Proceedings of the Second Annual Conference on Communication Networks and Services Research; Fredericton, NB, Canada, 19–21 May 2004; pp. 305-314.

29. Manaskasemsak, B.; Rungsawang, A.; Yamana, H. Time-weighted web authoritative ranking. Inf. Retr.; 2011; 14, pp. 133-157.

30. Citation Network Dataset: DBLP+Citation, ACM Citation Network. Aminer. Available online: https://www.aminer.org/citation (accessed on 11 August 2024).

31. Tang, J.; Zhang, J.; Yao, L.; Li, J.; Zhang, L.; Su, Z. ArnetMiner: Extraction and Mining of Academic Social Networks. Proceedings of the Fourteenth International Conference on Knowledge Discovery and Data Mining; Las Vegas, NV, USA, 14–18 August 2008; pp. 990-998.

32. DBLP. Computer Science Bibliography. Available online: https://dblp.org/ (accessed on 11 August 2024).

33. Association for Computing Machinery. Available online: https://www.acm.org/ (accessed on 11 August 2024).

34. Microsoft Academic Graph. Available online: https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/ (accessed on 11 August 2024).

35. Sinha, A.; Shen, Z.; Song, Y.; Ma, H.; Eide, D.; Hsu, B.-J.; Wang, K. An Overview of Microsoft Academic Service (MAS) and Applications. Proceedings of the 24th International Conference on World Wide Web (WWW ’15 Companion); ACM, New York, NY, USA, 18–22 May 2015; pp. 243-246.

36. Kuchanskyi, O.; Andrashko, Y.; Biloshchytskyi, A.; Omirbayev, S.; Mukhatayev, A.; Biloshchytska, S.; Faizullin, A. Gender-Related Differences in the Citation Impact of Scientific Publications and Improving the Authors’ Productivity. Publications; 2023; 11, 37. [DOI: https://dx.doi.org/10.3390/publications11030037]

37. Lee, K.; Park, I.; Yoon, B. An approach for R&D partner selection in alliances between large companies, and small and medium enterprises (SMEs): Application of Bayesian network and patent analysis. Sustainability; 2016; 8, 11. [DOI: https://dx.doi.org/10.3390/su8020117]

38. Salton, G.; McGill, M.J. Introduction to Modern Information Retrieval; Mcgraw-Hill College: New York City, NY, USA, 1983.

39. Biloshchytskyi, A.; Kuchanskyi, O.; Andrashko, Y.; Mukhatayev, A.; Toxanov, S.; Faizullin, A.; Yurchenko, N. Revealing the closeness of publication ties in scientific cooperation taking into account scientific productivity based on the Time-Weighted PageRank Method with Citation Intensity. East.-Eur. J. Enterp. Technol.; 2024; 5, pp. 63-70. [DOI: https://dx.doi.org/10.15587/1729-4061.2024.312884]