A methodology to infer value networks from police case files

Abstract

Every year, hundreds of case files from police investigations and court cases are produced. The challenge of manual processing and analysing the vast volume of documentation resulting from investigations have prompted law enforcement agencies to embrace Natural Language Processing (NLP). Despite this, a systematic approach to automatically retrieve relevant data from police case files is lacking. To address this gap, we propose a methodology to process police case files and perform criminals’ role classification, criminals’ ties classification, and value network inferring through syntactic pattern recognition and BERTopic. The results reached allow us to affirm that the inferred value networks can congruently depict the roles and the connections between them. Although further research is needed, this methodology can propel the automation of the extraction of criminal networks’ dynamics implemented by law enforcement agencies, allowing them to refine and improve the strategies employed to disrupt criminal networks.

Introduction

Over the last decades, the Netherlands observed a massive growth of criminal organizations (EUROPOL, 2021). Local, European, and Latin American organizations operating in the Netherlands have expanded by importing and selling large quantities of drugs, taking advantage of Dutch infrastructures, particularly the Port of Rotterdam (Roks et al., 2021). To execute the drug trade effectively, extended value networks consisting of dozens of roles, such as financial specialist, transporter, and couriers, must be established and maintained by criminal organizations (Oetker et al., 2023). The persons fulfilling the roles as well as their connections in the value network change over time, and often intersect with other value chains, making it complex and time-consuming for law enforcement to maintain a good overview (Roelofsen et al., 2023).

To deal with the outlined scenario, law enforcement agencies are adopting data-driven approaches. Social Network Analysis and Intelligence Gathering aim to analyze criminals’ behaviours and interactions to find the best tactic to disrupt them (Duijn and Klerks, 2014; Oetker et al., 2023, Roelofsen et al., 2023; Duijn et al., 2014; Duijn and Sloot, 2015). These approaches rest on information retrieved from investigation-derived documents, such as police case files.

Notwithstanding the abundance of police and witness reports and the implementation of NLP methodologies to extract information from police and witness reports (Chau et al., 2002; Ku et al., 2008; Porumb et al., 2023), a systematic approach based on NLP to automate the data extraction from police case files at this moment is lacking. Although previous methodologies accomplished the extraction of criminals’ relations, to the best of our knowledge there exists no paper that deals with the extraction of criminals’ roles from text data.

Thus, we propose a methodology based on topic modelling and syntactic pattern recognition to automate the classification of criminals’ roles and the extraction of criminals’ ties, to infer value networks, thus showing how the different roles in drug trafficking networks are connected.

Related literature

In the field of NLP, researchers extensively applied relation extraction in diverse scientific realms, including Criminology. Crime scripting has been an integral part of the social network approach for the last two decades (Calderoni, 2010), where role classification and value network inferring have been a manual endeavour. The reason for this is that role attribution as compared to relationship extraction is more heavily reliant on complex qualitative data, often written for criminal investigations instead of social network research. At the same time, network researchers are confronted with increasing amounts of qualitative law enforcement data that is too time-consuming to manually analyze. Moreover, although law enforcement agencies are increasingly relying on computational approaches, they still rely on the manual extraction of necessary information on which to run the models, thus making these systems not scalable. In this paper, we present a novel approach to make these systems scalable, by not only automating role and relationship extraction but also the extraction of information which is needed to perform the classification.

Relation extraction

Here we present the most relevant research concerning relation extraction implemented in the criminological (Ku et al., 2008) employed Information Extraction (IE) techniques and cognitive interviews, to retrieve criminal-related entities from witness reports, and Chau et al. (2002) developed an approach based on lexical lookup, ML, and minimal hand-crafted rules to extract meaningful entities from police narrative reports. The papers represent attempts to extract entities from text documents produced by law enforcement agencies, which as will be seen in the methodology section inspired us to develop a rule-based approach to implement relation extraction. Porumb et al. (2023), implemented a rule-based approach to retrieve, from Dutch court cases, drug traffickers’ Modus Operandi (MOs), which correspond to the modalities that drug traffickers follow to commit their crimes. To implement this approach, the researchers employed sentence parsing, which allowed them to extract from each sentence in court cases the subjects, the locations, the time and the actions characterising the crime. Once information was extracted they inferred statistics and trends about past crimes.

Although past work has examined the problem of relation extraction concerning law-enforcement documents, to the best of our knowledge there is lacking of a rule-based approach which can be employed with a low amount of data, tailored specifically to the entities and relations found in police case files.

Role classification

Considering the importance of classifying the roles and tasks performed by individuals to understand goal-driven social networks (Zygmunt, 2017), we present the most relevant papers found in the context of role classification. Kazmi et al. (2018) divide role detection approaches into three categories: graph, feature, and hybrid methodologies. In the field of social networks (Rahimkhani et al., 2015; Zhou et al., 2016), and Fu et al. (2015) proposed graph-based approaches to respectively classify nodes in the groups of Leaders (L), Communications (C), and Members (M), finding the most people that maximize the spread of influence in a social network and identify the most influential nodes. Bian et al. (2017) proposed a feature-based approach to quantify the influence of each node. In the criminal field, Taha and Yoo (2017) and Berzinji et al. (2012) propose graph-based approaches to implement role classification in criminal networks. Husslage et al. (2015) implemented a hybrid-based approach that accounts for network topology and individual and coalition features to rank terrorists in Al-Qaeda’s 9/11 attack.

Among the papers described above, four of seven papers study role classification in fields different from Criminology and leverage network topology in the first place. Moreover, in all the cited articles automated information retrieval from text is absent. Thus, to the best of our knowledge, there is lacking of an approach which solves the problem of role extraction in the criminal field by direct extraction from textual data.

[See PDF for image]

Fig. 1

Automation pipeline. The first step consists of giving as input the text contained in the case files to the pipeline and the extraction of the criminals’ behaviours. With criminals’ behaviours, we refer to those sentences which refer to the actions and the events characterising a specific criminal actor. Over the second step, the sentences representing actors’ behaviours are pre-processed. Over the third step, the pipeline performs the extraction of the ties and the classification of the criminals’ roles, and by leveraging this information it infers the values networks

Value networks inferring

Scientific research on value networks is a relatively new field and is starting to accompany the research centred on value chains. Value networks show the importance that different individuals and organizations possess when interacting with their membership groups. An example of this accompaniment and transition is presented in the paper published by Peppard and Rylander (2006), who introduces the value network concept. The authors conceptualize Network Value Analysis (NVA) as a way to analyze competitive ecosystems, an alternative to the notion of a value chain.

Helfen et al. (2018) propose another paper centred on the transition from a value-chain perspective to a value network. Of relevant interest, given the text feature extraction employed, is the paper published by Li et al. (2019), where the authors model value networks characterizing the manufacturing industry. In the realm of Computational Criminology, it is worth considering the papers written by Duijn et al. (2014) who introduce the concept of value network, and Oetker et al. (2023), who integrates the concept of value network to propose FREIDA (Framework for Expert-Informed Data-driven Agent-based models) to establish a modelling procedure for computational criminal modelling, and by Roelofsen et al. (2023) who implements an interactive visualization and simulation environment for exploring criminal network dynamics using computational models.

Though this research focuses on value networks, other concepts from the field of criminology and network analysis are helpful to understand and contextualize our proposed innovation. For example, in ego networks, the relations of a single individual and the chosen degree around them are investigated. Bright et al. (2015) used this approach to investigate how social networks facilitate drug trafficking and gang activity by analyzing the characteristics and criminal involvement of ego networks. Apart from ego-networks, techniques such as network brokerage analysis (NBA, examining individuals functioning as bridges between communities and facilitating information exchange or criminal activity) (Carpenter and Cooper, 2015; Gould and Fernandez, 1989), network flow analysis (NFA, tracking the flow of resources such as drugs throughout the network and identifying critical pathways (Zakimi et al., 2023) or dynamic network analysis (DNA), the change of networks over time, allowing for the analysis of the impact of interventions (Snijders et al., 2010) can be employed.

From here, we can draw a connection to value chains and value networks (VC and VN), in which agents that are dependent on each other are connected (also called dependency networks). However, Peppard and Rylander argue that the traditional value chain model might not fully capture the complex interactions within a network. Their concept of a value network, with its emphasis on collaboration and partnerships, as demonstrated in mobile operators, can be applied to non-ideology-driven criminal networks as well, as Criminal networks involve collaboration between different actors, each contributing specific resources or skills. Mobile operators, Peppard and Rylander (2006) explain, must initiate and upkeep a flow of value within their networks. Similarly, criminal networks rely on coordination to ensure fruitful operations. We must consider that while mobile operators as well as business-oriented criminal networks focus on economic value creation, criminal networks can not fully operate as businesses would, given that they need to consider the security/efficiency trade-off within their operation Morselli et al. (2007).

Data and methods

In this section, we describe the dataset and the methodology developed to implement role classification, relation extraction, and value network inferring. Figure 1 details the workflow. The sentences describing the behaviours of criminals are extracted from the case files, processed, and stored.1 We implement role and relation classification with the stored information. Using the classified roles and ties, we iterate through each actor in the case files and use the predicted roles and connections to build the value network. We evaluate the classification of roles and ties extraction over accuracy, precision, recall, and f1-score and value network inferring confronting the degree distribution and the geodesic distances of automatically and manually extracted value networks.

Datasets

We base this paper upon data granted by the Dutch police. Data consists of ten case files, originally written in Dutch, where each case file is composed on average by $\sim$120 sentences narrating the events related to a specific criminal network, composed of between four and ten criminals who interact over a maximum of two years (see Fig. 5 in the Appendix to visualize a case file). The evolution is divided into a maximum of four timesteps from $t_0$ to $t_3$. $t_0$ corresponds to either a police operation or homicide internal to the network, $t_1$ stands for one month after the intervention, $t_2$ for months or years after the main event, and $t_3$ for years after the disruptive event. Other than the case files, the Dutch police provided us with three datasets. These constitute the manually extracted ground truth used to validate our methodology.

Roles. This dataset contains, for each case file, for each timestep, the criminals and their associated roles. For each timestep, each actor can be assigned one of seven different roles. See Table 9 and Fig. 6 in the Appendix to overview the description of the roles and the distribution of roles across the dataset, respectively.

Relations. This dataset contains, for each case file and for each actor involved at timestep $t_0$, the sentences expressing the relations between actors, the entities involved, and the type of relation. To overview descriptions of the relations and the distribution of relations across the dataset, see Table 10 and Fig. 7 in the Appendix to respectively overview the type of possible relations and the distribution of types of relations across the dataset.

Value Networks. This dataset contains, for each case file, a graph constituting the ground-truth value network, where each node corresponds to a criminal actor with its role and each edge to a relation. For each case file, the dataset combines criminals’ roles and connections at timestep $t_0$. The networks were assembled by members of the Amsterdam Police, specifically of the Team Criminele Inlichting (TCI).

Automation pipeline

Figure 1 describes the automation pipeline that extracts value networks from police case files. The pipeline is characterized by three steps. The first step consists of the extraction of criminal behaviours corresponding to the sentences that describe criminals’ actions, the second step pre-processes the sentences, and the third step operates the role classification of each criminal and the extraction of their relations, which are then used to generate the values networks.

Data processing

In the following subsections, we describe the operations concerning the pre-processing of the case files, contained in step one and step two in Fig. 1.

Document Translation Since the models employed to implement the role classification are trained on English-language datasets, we translate the documents from Dutch to English. The translation of the documents is entrusted to experts who have deep knowledge of both Dutch and English and can transfer both the lexical and the syntactical content from Dutch to English, limiting the performance loss that models trained on native-English texts could suffer from being trained on wrongly translated documents.

Extraction of criminals’ behaviours

Since we base the role and tie classification on criminals’ behaviours, we retrieve the sections representing the timesteps of the networks that contain criminals’ behaviours, with the word behaviours indicating the actions committed by the criminals. For each timestep, we split the paragraph related to a specific actor into compound sentences, thus isolating the singular events regarding that specific actor.

[See PDF for image]

Fig. 2

The figure shows the two-dimensional projection of the clusters that populate the topic space. We build the clusters leveraging the pre-trained BERTopic model all-mpnet-base-v2. Each dot represents a sentence which corresponds to the behaviour or event connected to a specific criminal. The dots of the same colour belong to the same cluster. The words describing each cluster are the ones with higher frequency and originate from the elements of the cluster. The two clusters used to implement role classification consist of the 5 th and 6 th, which respectively refer to sentences related to cocaine transport, money laundering, and violent jobs. The 1 st and 7 th clusters group the sentences that do not contain any noteworthy event or behaviour and are composed of subject, verb to be, conjunctions, and propositions, which were not deleted during the preprocessing. We decide to keep stopwords because from the several iterations made we saw that by removing stopwords the model is less able to generate defined topics

Role classification

This phase is implemented as part of step three in Fig. 1. We compare two macro approaches: combining syntactical pattern recognition and BERTopic (Grootendorst, 2023a), and employing BERTopic alone. See Sect. “BERTopic” in the Appendix, for the definition of BERTopic and for details regarding its functioning and composition.

Combined approach

Syntactical pattern recognition The syntactical phase involves searching for core expressions in the actors-related sentences, through a comparison of sentences’ words. Core expressions explain the roles or operations for which actors are responsible. This phase comprises an explicit and implicit search. The first type of search identifies expressions that unambiguously declare the function of the criminal, while the implicit search tallies the occurrences of role-associated linguistic utterances in the sentences related to the actor. If the case file does not explicitly state the role we employ the implicit approach. To implement this, we manually gather core expressions from the case files’ used for training. Specifically, we focused on the sections regarding the timesteps of network evolution, which gather the criminals’ behaviours. Manually analyzing those sections we were able to collect the core expressions and assign them the corresponding role. We search for these expressions in the test case files, counting for each actor how many times the stored patterns appear in the text. In the end, the algorithm counts how many times the roles associated with the counted patterns appeared, and the role that appeared the majority of times is assigned to the actor.

Syntactical phase with embeddings To enhance the flexibility of implicit pattern recognition, we introduce a variant that replaces the strict search for key expressions with a comparison between actor-related sentences and manually collected utterances. We implement this comparison using cosine similarity. Cosine similarity is a metric that measures the correlation between two words or sentences, and since when working with Sentence Transformers words and sentences are represented through vectors, cosine similarity measures to what extent two vectors have the same orientation and thus correlation. The algorithm compares each clause associated with a particular actor to the gathered core expressions. For each comparison, the algorithm calculates the cosine similarity. If the cosine similarity exceeds the threshold derived from training case files, the methodology deems the clause to express a function associated in the dictionary with a particular role. As in the case of pattern recognition, the role that appears most frequently is considered the actor’s role. The introduction of cosine similarity adds flexibility because it allows a successful comparison when the sentence in the text is composed of synonyms of the words in the core expression. At the same time since all-MiniLM-L6-v2, the model used to generate the embeddings, is tuned on general-context data, it calculates a high cosine similarity also for words which in the criminal context should not be taken as synonyms (e.g. the cosine similarities calculated between he is a murder broker and, respectively, he is hired as a murder broker and he is hired as an assassin, tend to be the same). Thus, if there are two or more role categories with the same amount of sentences, we choose the category with the highest overall cosine similarity. If all the categories have the same cosine similarity, we randomly select the definitive role of the actor between the categories with the highest cosine similarity.

Topic modeling phase In case the syntactical phase fails role classification, we employ topic modelling. To deal with the scarcity of data, we use pre-trained BERTopic models. The prediction of the role consists of three steps:

1. Clustering space generation: in Fig. 2 we can see the clusters generated fitting the BERTopic model on the actor-related.

2. Cluster selection: between the eight generated clusters we only use two to classify roles: cluster number six and number five, which respectively assemble sentences related to violent roles, such as Murder Broker, Person of Violence, and organizational roles such as Financial Specialist, Exporter, and Organizer. Only these two clusters are chosen because they are the only two role-related clusters. To generate more clusters, the model should be fitted on more data. See Table 9 in the Appendix to overview the description of each role.

3. Role prediction: the model for each actor and each sentence in case files predicts to which cluster it belongs. For each of the two relevant clusters, the algorithm counts the number of utterances the model has assigned to each of them, and the cluster with the highest number of sentences is selected to represent the role. If the same number of sentences is assigned to each cluster, the algorithm uses the average confidence with which the BERTopic model has predicted the sentences to belong to the topic groups and considers the maximum.

Topic modeling phase—guided

A variant of the first topic modelling version is guided topic modelling. We define three lists of words called seeds and input them into the model. For each seed the model tries to build a topic, grouping the sentences that contain the words listed into the seed. Training the model through this technique generates an additional cluster titled 4 role kingpin normally that groups clauses related to kingpins (see Fig. 8 in the Appendix to visualize the generated topics).

The topic modelling phase allows for a macro-distinction between organizational functions and violent roles, thus not reaching the granularity of the pattern recognition phase. Due to this reason, we employ it only if the first phase fails the classification. If the topic modeling phase fails role classification the algorithm declares the role unknown.

Relation extraction

This phase is implemented as part of step three of Fig. 1. To operate the extraction of criminals’ relations, we experiment with two macro approaches: a rule-based approach and two ML models. Once the algorithm concludes the extraction of ties explicitly represented in the text, we infer transitive brotherhood relationships that are not explicit in the text.

Embeddings approach

We base the embeddings approach on embeddings comparison. The algorithm, for each actor, loops over the compounded sentences generated during the phase of extraction of criminals’ behaviours checking for each sentence if two actors are present. If only one actor is present in the sentence, then the sentence is discarded as it cannot convey any relation between actors. If the result is positive, the algorithm calculates the cosine similarity between the behaviour under consideration and a list of sentences of reference which represent relationships. The type associated with the sentence that shares with the criminal behaviour the highest cosine similarity is stored as the type of the relationship. To determine the direction of a relationship, we extract the subject and object of the sentence. This allows us to identify where the relationship originates and where it leads, effectively establishing its direction.

Here we list the ground truth sentences with which each behaviour is compared to check the type of the relationship: ’contact of’, ’murder broker of’, ’related to’, ’brother of’, ’son of’, ’sister of’, ’cousin of’, ’childhood friend’, ’former violence man of’, ’receives orders from’, ’is in contact with’, ’friends of’, ’befriended’, ’communicates with’. We select a minimum threshold for the cosine similarity that allows to assign a relation type to a specific relation between two actors (see “Relation extraction” subsection to check its specific value). If all the cosine similarities calculated are below the threshold, the relation is marked as being of type unknown.

ML-based approach

To implement the ML approach, we independently train and test Support Vector Machines (SVMs) (Cortes and Vapnik, 1995) and Random Forests (RFs) (Breiman, 2001) models on a dataset of sentences representing relationships labelled with the type of relationship they represent (see Sects. “Support vector machines” and “Random forests in the Appendix for details regarding the SVM and RF algorithms respectively). We build the embeddings using the word2vec approach, which allows us to capture words’ semantic relationships while improving computational performance by applying dimensionality reduction.

Inferring of transferable relations

Once we calculate the ties for all the criminals, we infer relationships that are not directly retrievable from the text. Between transferable relationships, we focus only on the transitive fraternal relationships, since other relationships such as friendship are not transitive by definition. We employ an algorithm that builds for each actor a set composed of their brothers and sisters. For each pair of actors, it calculates the intersection of the computed sets, and if it is not empty, it concludes that those two actors have brothers or sisters in common. If the fraternal relationship of the couple under analysis is absent from the adjacency matrix, the algorithm adds it.

Value network inferring

This is the final stage of step three in Fig. 1. We infer value networks by combining the information extracted during role and relation classification. For each actor, the algorithm isolates the role predicted at timestep t0 and generates a node with as a feature the actor’s role. The nodes within the case file under analysis are connected through the edges predicted during the relation extraction. The result shown in Fig. 3 is the graph corresponding to the value network for a specific criminal group. It shows the attribute role of each agent and thus which roles are required for the functioning of a drug trafficking network. To quantify the similarity between the inferred networks and dataset networks we calculate and compare the node degree distribution (Karyotis and Khouzani, 2016) of the networks by employing a goodness-of-fit test, a statistical hypothesis test used to see how closely observed data mirrors expected data. Moreover, we compare the networks by calculating and comparing the average geodesic distance (Han et al., 2012) between all the role pairs in inferred and dataset networks. We employ node degree distributions to analyze and understand the characteristics of networks including whether both networks have many nodes with similar degrees or a few highly connected hubs. At the same time, comparing geodesic distances (shortest path lengths) between roles helps in understanding to what extent the generated networks replicate the role distances characterising dataset networks.

[See PDF for image]

Fig. 3

Value network. The figure shows the value network inferred from case file b for timestep t0. The node grade represents the number of connections: yellow and red nodes have the lowest and the highest degree, respectively. The kingpin possesses the highest degree

Evaluation setup

For both role and tie classification, we use a random train/test split employing seven case files for training and three case files for testing, where each case file has a length corresponding to $\sim$120 sentences. Shuffling actors and relations from multiple files is strictly prohibited. We define that the sentences belonging to a specific document will only be used for training, to ensure that the model is able to generalise from actors contained in specific case files to the actors characterising another case file of which sentences are not contained in the training dataset.

Role classification

To implement the methodology, we gather the core role-recognizing expressions, compute the cosine similarity threshold for embedding comparison, and choose the best hyper-parameters to fit the BERTopic model to the data. We use seven case files to gather the core sentences, compute the cosine similarity threshold and select the hyperparameters for the pre-trained model, and three case files to test the methodology. To choose the best cosine similarity threshold, we check model performance with values between 0.50 and 0.95 and register that the best minimum limit is 0.55.

Hyperparameters. Given the modularity of BERTopic, we tested different hyperparameters, obtaining the best result by combining the pre-trained model all-mpnet-base-v2 to generate the vectors representing the sentences, Uniform Manifold Approximation and Projection (UMAP) and Hierarchical Density-based spatial clustering of applications with noise (HDBSCAN) (McInnes et al., 2017, 2018) to respectively perform dimensionality reduction and clustering, and CountVectorizer and c-TF-IDF (Grootendorst, 2023b) to get the description of the topics (see the Appendix, sect. “BERTopic decomposition” for the details regarding these algorithms). Relating to embedding comparison, the best embedding model is all-MiniLM-L6-v2. To choose the best cosine similarity threshold, we check model performance with values between 0.50 and 0.95 and register that the best minimum limit is 0.55.

During the hyperparameter selection, we also experiment with different methods to represent topics. We set the parameters bm25_weighting and reduce_frequent_words to True to reduce the impact of words that appear too frequently. The first parameter helps the model to be more robust against the stop words in small datasets, while the second helps reduce the impact of words that appear too frequently. Enabling these modalities does not help in improving the topic generation.

Relation extraction

We apply the same approach to extract relations for the rule and the ML-based procedure. We test the rule-based and embedding-based approach by employing only the core expressions retrieved from the seven training case files. The seven training case files are also used to define the minimum cosine similarity threshold to assign a specific type of relation. We check the performance with threshold values between 0.50 and 0.95 and register that the best minimum threshold is 0.60.

We implement the ML approach training and testing the SVMs and RFs models with the relations dataset, using the sentences from seven case files as training and the sentences from the remaining three case files as test dataset. Relation extraction only considers timestep t0.

Value network inferring

To evaluate the network inferring, we compare the degree distribution characterising inferred networks and dataset networks. Moreover, to gather further insights into the capacity of the pipeline to generate reliable networks, for each role pair we compare the geodesic distance in the inferred networks with the dataset networks. Since the dataset only contains value networks for timestep t0, the inferring is implemented only for the first timestep.

Evaluation metrics

In Eqs. 1, 2, 3, 4, and Table 1, we list and describe the metrics employed to evaluate the best BERTopic model, compare the inferred value networks to the manually extracted networks, and test the proposed methodologies.

Accuracy, described in Eq. 1, is the proportion of correct predictions, concerning a specific role of relation category, done by the model.

1

$$\begin{array}{c}\hfill Acc=\frac{TP+TN}{TP+TN+FP+FN}\end{array}$$The precision, described in Eq. 2 is the metric that for a specific role expresses the actors of that role that have been detected.

2

$$\begin{array}{c}\hfill Pr=\frac{\mathit{TP}}{TP+FP}\end{array}$$The recall, described in Eq. 3 is the percentage of known actor or connections of a specific category retrieved.

3

$$\begin{array}{c}\hfill Rec=\frac{\mathit{TP}}{TP+FN}\end{array}$$F1-score, described in Eq. 4, is the metrics that express the balance between precision and recall.

4

$$\begin{array}{c}\hfill F1=2·\frac{Pr·Rec}{Pr+Rec}\end{array}$$

Table 1. The table describes the metrics used to compare inferred and dataset networks of each actor and role

Metric	Definition
Degree distribution	Given that the degree of a node in a network is its number of connections, the degree distribution is the probability distribution of the degrees in a network.
Geodesic distance	Considered two vertices, the geodesic distance is the path of minimum length between them.

Accuracy, precision, recall, and f1-score are employed to choose the best hyperparameters for the BERTopic model. To do this we try different configurations of hyperparameters when executing the topic modelling phase and select the configuration that guarantees the best accuracy, precision, recall, and f1-score. The same process is executed for the cosine similarity threshold, and to choose the best approach to perform relation extraction. We consider precision, recall, and f1-score in their weighted variant, which allows calculating the average metrics accounting for the number of samples of each category. Moreover, the parameter zero_division is set to 0, such that if there is a division per 0, the value given to the metric will be 0, avoiding overestimating the model performance. In each table values in bold represent the maximum performance reached for a specific metric.

Table 2. Accuracy and average precision, recall, and f1-score for role classification implemented with guided BERTopic alone, and BERTopic both guided and unguided combined with static syntactic pattern recognition and embedding comparison

	BERTopic (guided)	BERTopic + Pattern recognition	BERTopic (guided) + Pattern recognition	BERTopic + Embeddings	BERTopic (guided) + Embeddings
Accuracy	0.68	0.85	0.85	0.83	0.81
Precision	0.64	0.88	0.86	0.87	0.85
Recall	0.68	0.85	0.85	0.83	0.81
F1-score	0.66	0.84	0.84	0.83	0.81

Baseline

Role Classification The topic modelling phase based on guided BERTopic is the baseline for role classification. Other than guided BERTopic, we considered unguided BERTopic. We dismissed the unguided BERTopic model because it could not generate the Kingpin cluster essential for classification, either due to lack of data or lack of guidance in its generation as it happens in the guided BERTopic model.

Relation Extraction The baseline for relation classification is SVMs and RFs, widely used in literature to extract and classify relations in NLP.

Results

In this section, we present the results related to the classification of role and ties and value network inferring.

Role classification

To implement role classification, we experiment with five approaches: guided BERTopic and combining both unguided and guided BERTopic with pattern recognition and embedding comparison.

Guided BERTopic

Table 3. The table contains precision, recall, and f1-score calculated for each role through guided BERTopic

	BERTopic (guided)
	Precision	Recall	F1
Kingpin candidate	0.50	0.50	0.50
Person of Violence	0.69	0.75	0.72
Financial Specialist	0.00	0.00	0.00
Kingpin	0.55	0.67	0.60
Organizer	0.22	0.40	0.29
Murder broker	0.00	0.00	0.00
Exporter	0.00	0.00	0.00
Unknown	0.82	0.84	0.83

Table 2 and Table 3 show baseline results. Table 3 shows that the model performs best in the categories Unknown and Person of Violence. It is relevant that the model is unable to identify actors belonging to the category of Financial Specialist, Exporter, and Murder Broker.

BERTopic combined with pattern recognition and embeddings comparison

Table 4. The table contains precision, recall, and f1-score calculated for each role class for both BERTopic and guided BERTopic combined with static syntactic pattern recognition

	BERTopic + Pattern recognition			BERTopic (guided) + Pattern recognition
	Precision	Recall	F1	Precision	Recall	F1
Kingpin candidate	0.75	0.75	0.75	1.00	0.25	0.40
Person of violence	1.00	1.00	1.00	0.79	0.92	0.85
Financial specialist	1.00	1.00	1.00	1.00	1.00	1.00
Kingpin	1.00	0.67	0.80	1.00	0.78	0.88
Organizer	0.50	0.40	0.44	0.33	0.40	0.36
Murder broker	1.00	1.00	1.00	1.00	1.00	1.00
Exporter	1.00	1.00	1.00	0.00	0.00	0.00
Unknown	0.89	0.96	0.92	0.88	0.94	0.91

Table 5. The table contains precision, recall, and f1-score calculated for each role class, for both BERTopic and guided BERTopic combined with embeddings comparison

	BERTopic + Embeddings			BERTopic (guided) + Embeddings
	Precision	Recall	F1	Precision	Recall	F1
Kingpin candidate	0.44	0.75	0.75	0.44	1.00	0.62
Person of violence	0.73	1.00	1.00	0.71	1.00	0.83
Financial specialist	1.00	1.00	1.00	1.00	1.00	1.00
Kingpin	1.00	0.67	0.80	0.57	0.44	0.50
Organizer	0.75	0.40	0.44	0.75	0.60	0.67
Murder broker	1.00	1.00	1.00	1.00	0.67	0.80
Exporter	0.50	1.00	1.00	0.50	1.00	0.67
Unknown	0.92	0.96	0.92	0.95	0.84	0.89

Table 6. The table shows precision, recall, and f1-score, for each type of criminal relationship in the dataset, calculated with syntactic pattern recognition, SVMs, and RFs

	SVMs			RFs			Embeddings method
	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
Unknown	0.72	1.00	0.84	0.72	1.00	0.84	0.47	0.40	0.43
Friends	0.50	0.25	0.34	0.50	0.25	0.34	0.73	0.67	0.70
Brothers	0.00	0.00	0.00	0.00	0.00	0.00	0.82	0.60	0.69
Father and son	1.00	1.00	1.00	1.00	1.00	1.00	0.17	0.50	0.25
Kingpin and subordinate	0.00	0.00	0.00	0.00	0.00	0.00	0.38	0.43	0.40
Person of violence and boss	0.00	0.00	0.00	0.00	0.00	0.00	0.17	1.00	0.29
Murder broker and boss	0.00	0.00	0.00	0.00	0.00	0.00	0.17	1.00	0.29
Cousins	0.00	0.00	0.00	0.00	0.00	0.00	0.75	1.00	0.29

Table 7. Geodesic distances between role pairs

Role pairs	Minimum geodesic distance
	Inferred networks	Dataset networks
Kingpin—Man of violence	2.0	1.5
Kingpin—Kingpin candidate	1.0	1.0
Kingpin—Organizer	1.0	1.0
Kingpin—Murder broker	1.0	1.0
Kingpin—Financial man	1.0	1.0
Kingpin candidate—Man of violence	3.0	2.6
Kingpin candidate—Financial man	2.0	2.0
Kingpin candidate—Organizer	1.0	2.0
Organizer—Financial man	2.0	2.0
Murder broker—Man of violence	2.5	2.33

Tables 2, 4, and 5, show the results related to the combination of guided and unguided BERTopic, with pattern recognition and embeddings comparison.

It is important to notice that Table 2 shows that each combination between BERTopic and pattern recognition or embedding comparison outperforms the baseline constituted by guided BERTopic, in terms of accuracy, precision, recall, and f1-score.

The comparison between the combined models shows that unguided and guided BERTopic combined with pattern recognition reach the highest overall accuracy.

Table 2 shows that unguided BERTopic combined with pattern recognition and embedding comparison reaches the highest precision, while unguided and guided BERTopic combined with pattern recognition reaches the highest overall recall.

Significant is also the result given by guided BERTopic combined with embeddings comparison. Table 2 shows that in each metric it performs worse than the other models, thus being the worst model after the baseline.

Regarding specific categories in Table 4, it is relevant that guided BERTopic combined with pattern recognition, is completely unable to identify Exporter individuals.

Table 8. Accuracy and average precision, recall, and f1-score calculated for criminals’ ties classification, implemented with SVMs, Embeddings Comparison, and RFs

	SVMs	RFs	Embeddings Method
Accuracy	0.75	0.70	0.65
Precision	0.62	0.62	0.79
Recall	0.75	0.75	0.66
F1-score	0.66	0.66	0.71

Relation extraction

Tables 6 and 8 show the results of the relationship extraction.

Table 8 shows that the best approaches we employed are embedding comparison and SVMs. Embeddings comparison outperforms SVMs in precision and f1-score, while SVMs outperform Embeddings Method in accuracy and recall.

Table 8 shows also that RFs have the same performance as SVMs, apart in terms of accuracy, where they perform less.

Table 6 shows that both the ML models can identify when the type of relationship is Unknown, when the main actor in the sentence is the Son, or when the actors are Friends, but for all other categories for precision, recall and f1-score the score is 0.00.

Value network inferring

In this section, we present the results related to the capacity of the pipeline to infer value networks.

Degree distribution

We first focus on the degree distribution property. Figure 4, compares the degree distribution of inferred networks and dataset networks. The visual inspection leads us to hypothesize that both degree distributions follow the generalized normal distribution (Nadarajah, 2005; Wen et al., 2022). To test the truthfulness of the hypothesis developed upon visual evaluation, we employ the goodness-of-fit test. The null hypothesis of the test assumes that the sample data belongs to the generalized normal distribution. Assuming 0.05 as the threshold to reject the null hypothesis, considering that the test results present a p value = 0.29 for the distribution related to inferred networks and a p value = 0.16 for the distribution of dataset networks, which are both greater than the 0.05, we fail to reject the null hypothesis that the distributions follow the generalized normal distribution. The result thus indicates that the system is capable of replicating the connections which characterize the network.

[See PDF for image]

Fig. 4

This figure compares the degree distribution of inferred networks with the degree distribution of dataset networks. We notice that both distributions follow the shape of a generalized normal distribution. This is demonstrated by the p-values obtained during the goodness-of-fitness test

Minimum paths between roles

The second parameter of study that we consider to compare inferred networks and dataset networks consists of the geodesic distance. Geodesic distance expresses the shortest path between two vertices. To gather insights into the similarity between networks, for each pair of roles, we calculate the average geodesic distance in the inferred and dataset networks. Table 7, shows the results. It is possible to see that the inferred networks are capable of replicating the average geodesic distance for six role pairs. The highest difference to be registered is for the pair Kingpin candidate - Organizer, where the difference between inferred geodesic distance and dataset geodesic distance i 1.00. The minimum distance is instead seen for the pair Murder broker - Man of violence, where the discrepancy is 0.17.

Discussion

The outcomes of this research allow us to reflect on the application of topic modelling and pattern recognition techniques to extract and classify information contained in police case files, aiming at inferring value networks underlying criminal groups. We provide a comparison between the experimented approaches and dive into the limitations of the study.

Role classification

The results indicate that combining topic modelling and pattern recognition guarantees the best performance. Topic modelling alone does not allow for the extraction of Financial Specialist, Murder Broker, and Exporter actors because the model is not able to generate clusters related to those functions during the fitting process. The scarcity of role-related clusters is due to the low amount of sentences on which we tuned the guided BERTopic model. Combining pattern recognition and topic modelling allows us to fill the limitations posed by the lack of data. Inspecting the sentences that describe criminals’ actions, allows us to precisely identify actors’ functions that the pre-trained model cannot detect. If the performance given by the combination of unguided BERTopic with embeddings comparison is comparable to the combination of guided and unguided BERTopic, combining guided BERTopic with embeddings shows worse performance in each metric. Since the embeddings generated by the models to operate the confrontation are the same, this decay is possibly due to errors committed by the guided model, in assigning criminals’ behaviours to the correct cluster. As a result, it becomes evident that while guided BERTopic can generate an additional cluster related to kingpins, this impacts the overall formation of the topic space, resulting in inaccuracies in assigning behaviours. This challenge can be addressed either by utilizing the unguided model or, in the future, by training on larger datasets, which would effectively reduce outliers and enhance the generation of clusters.

Overall, our approach is successful in the extraction of criminal roles from police case files. While previous research focused on role extraction outside the realm of Criminology (Bian et al., 2017; Fu et al., 2015; Kazmi et al., 2018; Rahimkhani et al., 2015; Zygmunt, 2017), or accomplished the task by leveraging graph-based approaches (Berzinji et al., 2012; Husslage et al., 2015; Taha and Yoo, 2017), our approach is focused on the extraction of roles from police case files, relying directly on manipulation of the text characterising them and thus conveying direct information on the criminals.

Relation extraction

We find that the best approach to operating relation extraction consists of embedding comparison, outperforming SVMs, and RFs. We explain the poor performance of the baseline model and RFs considering the reduced size and the class imbalance characterising the dataset on which we train the models. These factors lead the ML models to perform poorly concerning the classification of relationships where the actors are Brothers or Cousins, or where the actors are Kingpin and subordinate, Boss and Person of Violence, Boss and Murder Broker.

Thus, while previous approaches (Chau et al., 2002; Ku et al., 2008) were ML-based, the embeddings approach is insensitive to the number of relationships of each type present in case files and, without any training, guarantees better performance. Thus, our method based on pre-trained models, can be considered a valid alternative to baseline models when the data used is not sufficient to train them.

Value network inferring

Focusing on value network extraction, inferred networks replicate congruently the dataset networks. This is indicated by the comparison of degree distributions, which indicate that both networks follow the same probability distribution in terms of node connections, and the geodesic distances which are equal for the majority of the role pairs indicated. Nevertheless, differences are also present, the degree distribution comparison indicates that in the inferred networks, it is more probable to have nodes with zero to three connections, while in the dataset networks is more probable that nodes possess four to five connections. These differences are produced by errors in the role and relation classification, which inevitably influence the quality of the inferring process.

While previous research initialized the field of the study of Value Networks (Duijn et al., 2014; Peppard and Rylander, 2006), and offered precious insights regarding the internal functioning of criminal networks (Morselli et al., 2007), our methodology based on NLP is the first approach that in the realm of Criminology allows for the automated extraction of criminal value networks from police case files, thus allowing to decrease the huge amount of work necessary to extract criminal networks from police documents.

Limitations

Although the results obtained applying the proposed methods on granted datasets are promising, limitations to their reliability, explainability, and generalizability exist.

Reliability

For classification of the role, the most relevant limit to the reliability of results is the dataset size because a low number of training samples accentuates the influence of outliers, and does not guarantee an adequate representation of the population. Concerning role classification, we combined pre-trained models and embeddings-based approaches to reduce the impact of dataset size on reliability. For the task of relation extraction, the reliability of SVMs, regarding the extraction and classification of entity relations from text, has been explored in past research with positive results. Thus it can be effectively used in our context. In the end, concerning value network inferring, limits to reliability exist due to the criticalities transitively passed by the role classification.

Explainability

Concerning role classification rule-based syntactic pattern recognition is a completely explainable approach because the underlying rules and patterns are explicit and designed by humans and data does not traverse any semantic transposition, which instead is a fundamental passage when using advanced architectures such as BERTopic which involves complex data transformations, limiting the explainability for humans of the underlying processes. With respect to relation extraction no criticalities exist regarding the explainability of SVMs, as past research has shown an acceptable degree of explainability regarding SVMs. Regarding value network inferring, the process of value network generation, once the roles and relations have been gathered, does not present any aspect that leads to a lack of explainability.

Generalizability

Regarding the task of role classification, given the use of rule-based approaches, generalizability is not guaranteed, because rule-based approaches require a continuous updating of the expressions that identify a role. Conversely, fitting BERTopic on a higher number of police case files, with a consequent generation of more specific clusters, would allow it to identify each role without pattern recognition, thus solving the lack of generalizability. For the extraction of relations, a positive side of SVMs is that it can generalize to any number of classes, without the need for manual upload. The limitation relies on the necessity to raise the number of training sentences in the dataset. Concerning value network inferring, the lack of testing on different investigation-derived documents limits the generalizability from two perspectives. The first perspective is technical because we cannot know whether the approach applied to other typologies of documents would return compliant results, and the second is criminological because we cannot compare the networks inferred from case files with networks derived from other types of reports, assessing their differences and similarities.

Limitation due to document translation

Another limitation which influences the entire pipeline is the necessity to translate the documents from Dutch to English, to guarantee the best performance of the pre-trained models used. Even if the translation is entrusted to officers who are specialised in this task, this step can negatively influence the model performance due to the presence of cultural and linguistic differences which cause loss of accuracy and contextual information during the translation process.

To deal with this limitation, it would be possible to encourage Dutch police and in general law enforcement agencies to train language models on documents written in their native language, thus eliminating the necessity to have the documents translated into English.

Conclusion and future work

Until now, researchers implemented role classification manually, leading to time-consuming endeavours, or they used automated topological methodologies leading to the extraction of only central actors such as kingpins or brokers. As a result, many studies are limited to small networks or cannot capture the overall network complexity. Due to the potential that NLP methodologies have to help in automatically classifying roles out of large qualitative datasets, opening the door towards studying the effect of value network dynamics on larger and more complex criminal systems, we propose a methodology based on syntactic pattern recognition and BERTopic, to perform criminals’ roles and ties classification and to infer value networks. The promising results suggest that the exploitation of NLP to automatically classify criminals’ functions and relations and infer value networks can aim at reaching the performance achieved through manual extraction and classification. Generated value networks truthfully replicate dataset networks in terms of degree distribution and shortest path between different roles.

However, despite promising results, reliability, explainability, and generalizability have several limitations. Concerning reliability, avenues for further research include the extension of the dataset, with the inclusion of more case files. Concerning the explainability-generalizability trade-off, the direction of future approaches should follow a decision in the realm of law enforcement agencies, clearing the priority between explainability and generalizability. If explainability is necessary for every aspect of the role and relation classification, future research should focus on the explainability of the employed approaches. Conversely, if law enforcement agencies deem generalizability of higher relevance researchers should follow the avenues of more complex topic modeling approaches. To improve the generalizability of the method presented to implement value network inferring, researchers should test it on different investigation-derived documents, such as police interviews and documents gathered from sources different from law enforcement. Testing the proposed approach on a wide variety of documents would allow researchers to check for the technical reliability and generalizability of the methodology and, in the criminological realm, verify if value networks derived from police case files are comparable to the ones derived from other types of sources. Also, it would allow to include data gathered from sources that possess a different and more complex vision than law enforcement. Moreover, even though the proposed pipeline can be successfully employed to perform criminal intelligence and criminal network analysis through uniquely NLP methods, future research could integrate NLP methods and topological methodologies such as the ones proposed by Lopes et al. (2022) and Ribeiro et al. (2023), to achieve better performances for role and relation retrieval, and computational models to allow for the complete automation of criminals’ network simulation.

Acknowledgements

We thank the Dutch police for providing the data that formed the basis of this manuscript.

Author contributions

Conceptualization: EM, FO, PD, RQ; Methodology: EM; Software: EM; Validation: EM, FO; Formal analysis: na; Investigation: EM; Resources: na; Data Curation: EM, FO; Writing - Original Draft: EM; Writing - Review & Editing: FO, PD, RQ; Visualization: EM; Supervision: FO, RQ; Project administration: FO, PD, RQ; Funding acquisition: PD, RQ. All authors read and approved the final manuscript.

Funding

This research was carried out within the context of the Hyperion Lab, IAS of the University of Amsterdam, and RIEC.

Data availability

The data that support the findings of this study are available from the Dutch police, but it cannot be disclosed due to its sensitive nature.

Declarations

Competing interests

The authors declare that they have no Conflict of interest. Authors’ contributions All authors read and approved the final manuscript.

1

In this manuscript, with the word sentence we indicate the sentences composing the text, that are generated by segmenting the corpus of the case files through pre-processing techniques.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

Berzinji, A., Kaati, L. & Rezine, A. (2012). Detecting key players in terrorist networks. In 2012 european intelligence and security informatics conference (p. 297–302).

Bian, T; Hu, J; Deng, Y. Identifying influential nodes in complex networks based on AHP. Physica A: Statistical Mechanics and its Applications; 2017; 479, pp. 422-436. [DOI: https://dx.doi.org/10.1016/j.physa.2017.02.085]

Breiman, L. Random forests. Machine learning; 2001; 45, pp. 5-32. [DOI: https://dx.doi.org/10.1023/A:1010933404324]

Bright, DA; Greenhill, C; Ritter, A; Morselli, C. Networks within networks: Using multiple link types to examine network structure and identify key actors in a drug trafficking operation. Global Crime; 2015; 16, pp. 219-237. [DOI: https://dx.doi.org/10.1080/17440572.2015.1039164]

Calderoni, F. Inside criminal networks, Morselli, Carlo, Springer, New York, 2009, 204 p. European Journal on Criminal Policy and Research; 2010; 2010, pp. 50-55.

Carpenter, A; Cooper, S. Understanding transnational gangs and criminal networks: A contribution to community resilience: A social network analysis of the San Diego/Tijuana border region. Journal of Gang Research; 2015; 22, pp. 1-24.

Chau, M., Xu, J.J. & Chen, H. (2002). Extracting meaningful entities from police narrative reports. Proceedings of the 2002 annual national conference on digital government research (pp. 1–5).

Cortes, C; Vapnik, V. Support-vector networks. Machine learning; 1995; 20, pp. 273-297. [DOI: https://dx.doi.org/10.1007/BF00994018]

Duijn, P; Sloot, P. From data to disruption. Digital Investigation; 2015; 15, pp. 39-45. [DOI: https://dx.doi.org/10.1016/j.diin.2015.09.005] Special Issue: Big Data and Intelligent Data Analysis

Duijn, PA; Kashirin, V; Sloot, P. The relative ineffectiveness of criminal network disruption. Scientific Reports; 2014; 4, 1 pp. 1-15. [DOI: https://dx.doi.org/10.1038/srep04238]

Duijn, PAC; Klerks, PPHM. Masys, AJ. Social network analysis applied to criminal networks: Recent developments in Dutch law enforcement. Networks and network analysis for defence and security; 2014; Springer International Publishing: pp. 121-159. [DOI: https://dx.doi.org/10.1007/978-3-319-04147-6_6]

EUROPOL. (2021). Cocaine insights 1 the illicit trade of cocaine from Latin America to Europe-from oligopolies to free-for-all? Retrieved February 20, 2021 from https://www.unodc.org/documents/data-and-analysis/cocaine/Cocaine_Insights_2021.pdf

Fu, Y-H; Huang, C-Y; Sun, C-T. Using global diversity and local topology features to identify influential network spreaders. Physica A: Statistical Mechanics and its Applications; 2015; 433, pp. 344-355. [DOI: https://dx.doi.org/10.1016/j.physa.2015.03.042]

Gould, RV; Fernandez, RM. Structures of mediation: A formal approach to brokerage in transaction networks. Sociological Methodology; 1989; 19, pp. 89-126. [DOI: https://dx.doi.org/10.2307/270949]

Grootendorst, M. (2023a). BERTopic: Leveraging BERT and c-TF-IDF for topic modeling.

Grootendorst, M. (2023b). c-TF-IDF (class-based TF-IDF).

Han, J., Kamber, M. & Pei, J. (2012). Advanced cluster analysis. In Data mining (third edition) (3rd ed., pp. 497–541). Morgan Kaufmann.

Helfen, M; Schüßler, E; Sydow, J. How can employment relations in global value networks be managed towards social responsibility?. Human Relations; 2018; 71, 12 pp. 1640-1665. [DOI: https://dx.doi.org/10.1177/0018726718757060]

Husslage, B; Borm, P; Burg, T; Hamers, H; Lindelauf, R. Ranking terrorists in networks: A sensitivity analysis of Al Qaeda’s 9/11 attack. Social Networks; 2015; 42, pp. 1-7. [DOI: https://dx.doi.org/10.1016/j.socnet.2015.02.003]

Karyotis, V; Khouzani, M. Karyotis, V; Khouzani, M. Applications of state-of-the-art malware modeling frameworks. Malware diffusion models for wireless complex networks; 2016; Morgan Kaufmann: pp. 181-214. [DOI: https://dx.doi.org/10.1016/B978-0-12-802714-1.00019-0]

Kazmi, F., Haider, W. & Saeed, A. (2018). Evaluation of role detection approaches in terrorist networks. (pp. 245–249).

Ku, C.H., Iriberri, A. & Leroy, G. (2008). Crime information extraction from police and witness narrative reports. 2008 IEEE conference on technologies for homeland security (pp. 193–198).

Li, J; Qian, C; Zhang, Y; Yang, H; Zhang, D; Ren, S. Modeling of the value network in smart manufacturing based on FAHP and text feature extraction. Procedia CIRP; 2019; 83, pp. 694-698. [DOI: https://dx.doi.org/10.1016/j.procir.2019.04.237]

Lopes, DD; Cunha, BR; Martins, AF; Gonçalves, S; Lenzi, EK; Hanley, QS; Ribeiro, HV. Machine learning partners in criminal networks. Scientific Reports; 2022; 12, 1 15746. [DOI: https://dx.doi.org/10.1038/s41598-022-20025-w]

McInnes, L; Healy, J; Astels, S. hdbscan: Hierarchical density based clustering. The Journal of Open Source Software; 2017; 2, 205. [DOI: https://dx.doi.org/10.21105/joss.00205]

McInnes, L., Healy, J. & Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprintarXiv:1802.03426

Morselli, C; Giguère, C; Petit, K. The efficiency/security trade-off in criminal networks. Social Networks; 2007; 29, 1 pp. 143-153. [DOI: https://dx.doi.org/10.1016/j.socnet.2006.05.001]

Nadarajah, S. A generalized normal distribution. Journal of Applied statistics; 2005; 32, 7 pp. 685-694. [DOI: https://dx.doi.org/10.1080/02664760500079464]

Oetker, F., Nespeca, V., Vis, T., Duijn, P., Sloot, P. & Quax, R. (2023). Framework for developing quantitative agent based models based on qualitative expert knowledge: an organised crime use-case.

Peppard, J; Rylander, A. From value chain to value network: Insights for mobile operators. European Management Journal; 2006; 24, 2 pp. 128-141. [DOI: https://dx.doi.org/10.1016/j.emj.2006.03.003]

Porumb, L., Munoz, J.P., Mezzi, E., Vaisanen, J. & Zeuzem, J. (2023). Finding mos in open-source court cases.

Rahimkhani, K., Aleahmad, A., Rahgozar, M. & Moeini, A. (2015). A fast algorithm for finding most influential people based on the linear threshold model. Expert Systems with Applications.

Ribeiro, HV; Lopes, DD; Pessa, AA; Martins, AF; da Cunha, BR; Gonçalves, S; Perc, M. Deep learning criminal networks. Chaos, Solitons & Fractals; 2023; 172, [DOI: https://dx.doi.org/10.1016/j.chaos.2023.113579] 113579.

Roelofsen, LAS; Oetker, F; van der Lely, M; Belleman, RG; Quax, R. Gillmann, C; Krone, M; Lenti, S. Citadelpolice: An interactive visualization environment for scenario testing on criminal networks. Eurovis 2023—posters; 2023; The Eurographics Association:

Roks, R; Bisschop, L; Staring, R. Getting a foot in the door. spaces of cocaine trafficking in the port of Rotterdam. Trends in Organized Crime; 2021; 24, pp. 171-188. [DOI: https://dx.doi.org/10.1007/s12117-020-09394-8]

Snijders, TA; Van de Bunt, GG; Steglich, CE. Introduction to stochastic actor-based models for network dynamics. Social Networks; 2010; 32, pp. 44-60. [DOI: https://dx.doi.org/10.1016/j.socnet.2009.02.004]

Taha, K; Yoo, PD. Using the spanning tree of a criminal network for identifying its leaders. IEEE Transactions on Information Forensics and Security; 2017; 12, 2 pp. 445-453. [DOI: https://dx.doi.org/10.1109/TIFS.2016.2622226]

Wen, L; Qiu, Y; Wang, M; Yin, J; Chen, P. Numerical characteristics and parameter estimation of finite mixed generalized normal distribution. Communications in Statistics-Simulation and Computation; 2022; 51, 7 pp. 3596-3620. [DOI: https://dx.doi.org/10.1080/03610918.2020.1720733]

Zakimi, N; Greer, A; Bouchard, M; Dhillon, A; Ritter, A. Sociometric network analysis in illicit drugs research: A scoping review. PLoS One; 2023; 18, 2 [DOI: https://dx.doi.org/10.1371/journal.pone.0282340] e0282340.

Zhou, M; He, X; Fu, Z; Zhuo, Z. Role extraction in complex networks and its application in control of networks. Physica A: Statistical Mechanics and its Applications; 2016; 442, pp. 458-466. [DOI: https://dx.doi.org/10.1016/j.physa.2015.08.035]

Zygmunt, A. (2017). Role identification of social networkers. (pp. 1–9).